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DESCRIPTION ^ *^ *^ ^WUU 

IMAGING APPARATUS 

TECHNICAL FIELD 

The present invention relates to an imaging 
apparatus that captures an image of a subject 
(hereinafter referred to as subject image) and stores 
the subject image on a storage medium, and more 
particularly, to an imaging apparatus including an 
image retrieving function. 



BACKGROUND ART 

Conventionally, an imaging apparatus that captures 
a subject image and stores the subject image on a 
storage medium has been put to practical use. However, 
as a capacity of the storage medium increases, it has 
been demanded that the stored image can be retrieved 
easily. 

In response to this demand, Japanese Patent 
Application Laid-Open No. 2000-76302 proposes an 
imaging apparatus that extracts a characteristic amount 
of image data stored on a storage medium and a 
characteristic amount of key image data and compares 
both the characteristic amounts to retrieve desired 
image data . 

There has also been proposed a method of making it 
easy to visually understand similarity of images 
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(Japanese Patent Application Laid-Open No. 2000-311246) . 
In the invention described in the reference, in 
displaying a similar image/ a characteristic amoiint is 
extracted from image data of respective images, a 
5 characteristic amount space is divided hierarchically 
on the basis of the extracted characteristic amount, a 
tree structure is formed, and a display space is formed 
on the basis of the tree structure. 

10 DISCLOSURE OF INVENTION 

However, image retrieving unit disclosed in the 
reference does not sufficiently take into account 
convenience for users in terms of imaging and setting 
of a key image, a management method for the key image, 

15 and the like. Therefore, there is plenty of room left 
for improvement in the image retrieving unit. 

For example, it would be preferable for a user if 
operation for capturing generic image data and 
operation for capturing key image data can be 

20 distinguished clearly without deteriorating consistency 
of operation. 

Incidentally, in an imaging apparatus that 
captures generic image data and key image data without 
distinction and stores these image data on a storage 

25 medium, if a large number of image data are stored on 
the storage medium, a user has trouble in work for 
selecting the key image data. In particular, in future. 
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it is expected that a capacity of the storage medium 
will increase rapidly and the number of image data 
stored on the storage medium will be enormous. Thus, 
the work for finding the key image data will be more 
5 difficult than in the past. 

Therefore, work for setting the key image data in 
image retrieving phase to be performed later would be 
easier if the generic image data and the key image data 
are distinguished and stored from the beginning at a 

10 phase of imaging. 

In order to solve the problems, the invention 
provides an imaging apparatus that images a subject 
image and stores the subject image on a storage medium, 
wherein the imagine apparatus images an generic image 

15 and a key image selectively. The invention provides an 
imaging apparatus that images a subject image and 
stores the subject image on a storage medium, wherein 
the imaging apparatus starts a retrieving operation 
using a key image with imaging of the key image as a 

20 trigger. The invention provides an imaging apparatus 
that images a subject image and stores the subject 
image on a storage medium, wherein the imaging 
apparatus stores a generic image on the storage medium 
and stores a key image on the identical storage medium, 

25 The invention provides an imaging apparatus that images 
a subject image and stores the subject image on a 
storage medium, wherein an image to be stored on the 



P205-0195WO 

- 4 - 

Storage medium is a generic image and can also be used 
as a key image. The invention provides an imaging 
apparatus that images a subject image and stores the 
subject image on a storage medium, wherein the imaging 
apparatus specifies an arbitrary image in the storage 
medium and reads out and uses the arbitrary image as a 
key image. The invention provides an imaging apparatus 
that images a subject image and stores the subject 
image on a storage medium, wherein an image specified 
as key information or an image appearing from a 
retrieval result is designated and a retrieval button 
is pressed to execute retrieval again with the image as 
a key image . 

The invention provides an imaging apparatus that 
images a subject image and stores the subject image on 
a storage medium, wherein an imaging operation for an 
image and a reproducing operation for an image is 
possible even during an image retrieval operation. The 
invention provides an imaging apparatus that images a 
subject image and stores the subject image in a storage 
medium, wherein, when the imaging apparatus is in modes 
other than an imaging mode, the imaging apparatus is 
always in a state in which an arbitrary retrieval key 
is selected as a retrieval key. The invention provides 
an imaging apparatus that images a subject image and 
stores the subject image on a storage medium, wherein, 
when the imaging apparatus selects a key image as a 
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retrieval key and displays images in an order of time 
or an order of file names. Programs of the respective 
inventions described above are operated in the imaging 
apparatus. The invention provides an imaging apparatus 
5 that images a subject image and stores the subject 
image on a storage medium, wherein the imaging 
apparatus images generic image data and key image data 
selectively- The imaging apparatus includes an 
operating unit that performs first operation input in 

10 acquiring the generic image data and second operation 
input in acquiring the key image data in different 
operation forms. 

The invention provides an imaging apparatus that 
images a subject image and stores the subject image on 

15 a storage medium, including: an imaging unit that 

performs imaging processing for key image data that is 
used as a retrieval key; and a retrieving unit that 
retrieves generic image data, which has a predetermined 
relation with the key image data, out of generic image 

20 data stored on the storage medium, wherein the 

retrieving unit executes retrieval processing using the 
key image data with completion of the imaging 
processing for the key image data by the imaging unit 
as a trigger. The invention provides an imaging 

25 apparatus that images a subject image and stores the 

subject image on a storage medium, wherein the storage 
medium is a detachably replaceable storage medium and, 
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when the storage medium stores image data stored in 
other apparatuses, the imaging apparatus further 
includes a selecting unit that selects at least one of 
the image data stored on the storage medium as key 
5 image data. The invention provides an imaging 

apparatus that images a subject image and stores the 
subject image on a storage medium, including: a 
retrieving unit that retrieves generic image data, 
which has a predetermined relation with key image data, 

10 out of generic image data stored on the storage medium; 
and a selecting unit that selects the generic image 
data retrieved by the retrieving unit as new key image 
data, wherein the retrieving unit executes retrieval 
processing again on the basis of the new key image data. 

15 The invention provides an imaging apparatus that 

images a subject image and stores the siibject image on 
a storage medium, including: an imaging unit that 
images generic image data to be retrieved; and a 
retrieving unit that retrieves generic image data, 

20 which has a predetermined relation with key image data, 
out of generic image data stored on the storage medium, 
wherein access to the storage medium by the imaging 
unit and access to the storage medium by the retrieving 
unit are executed by different controls such that, even 

25 when retrieval processing is executed in the retrieving 
unit, the imaging unit can image the generic imaging 
data in parallel with the retrieval processing. 
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The invention provides an imaging apparatus that 
images a subject image and stores the subject image on 
a storage medium, including: a retrieving unit that 
retrieves generic image data, which has a predetermined 
5 relation with key image data, out of generic image data 
stored on the storage medium; and a reproduction 
processing unit that reproduces the generic image data 
stored on the storage medium, wherein access to the 
storage medium by the reproduction processing unit and 

10 access to the storage medium by the retrieving unit are 
executed by different controls such that, even when 
retrieval processing is executed in the retrieving unit, 
the reproduction processing unit can reproduce the 
generic image data in parallel with the retrieval 

15 processing. 

The invention provides an imaging apparatus that 
images a subject image and stores the subject image in 
a storage medium, including: an operation mode setting 
unit that sets an operation mode of the imaging 

20 apparatus; and a retrieving unit that retrieves generic 
image data, which has a predetermined relation with key 
image data, out of generic image data stored on the 
storage medium, wherein, when the imaging apparatus is 
set in an operation mode, in which retrieval processing 

25 is executable by the retrieving unit, the retrieving 

unit maintains the key image data designated in advance 
as a retrieval key. 



I 



P205-0195WO 

- 8 - 

The invention provides an imaging, apparatus 
including: an imaging unit that images an image of a 
subject to acquire generic image data to be retrieved 
and key image data that is used as a retrieval key; a 
5 storage medium that stores the imaged generic image 
data; and a retrieving unit that retrieves generic 
image data, which has a predetermined relation with the 
key image data, out of the generic image data stored on 
the storage medium, wherein the imaging apparatus 

10 further includes a storage control unit that stores the 
key image data imaged by the imaging unit in the 
storage medium in an expression distinguishable from 
the general image data. 

The invention provides an imaging unit that images 

15 an image of a subject to acquire generic image data to 
be retrieved and key image data that is used as a 
retrieval key; an image storage medium that stores the 
imaged generic image data; a retrieving unit that 
retrieves generic image data, which has a predetermined 

20 relation with the key image data, out of the generic 
image data stored in the image storage medium; and an 
operating unit for inputting an operation input, 
wherein, when a first operation input in acquiring the 
generic image data is inputted from the operating unit, 

25 the imaging apparatus controls the imaging unit to 

acquire the generic image data and store the generic 
image data on the storage medium and, when a second 
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operation input for acquiring the key image data 
inputted in an operation expression form different from 
the first operation input is inputted from the 
operating unit, the imaging apparatus controls the 
5 imaging unit to acquire the key image data. 

The invention provides an imaging apparatus 
including: an imaging unit that images an image of a 
subject to acquire generic image data to be retrieved 
and key image data that is used as a retrieval key; an 

10 image storage medium that stores the imaged generic 
image data; a retrieving unit that retrieves generic 
image data, which has a predetermined relation with the 
key image data, out of the generic image data stored on 
the image storage medium; and an operating unit for 

15 inputting an operation input, wherein the imaging 
apparatus stores the key image data imaged by the 
imaging unit on the storage medium in an expression 
distinguishable from the generic image data. 

As explained above, according to the invention, it 

20 is possible to provide an imaging apparatus that can 
distinguish two kinds of imaging operations from each 
other clearly. According to the invention, it is 
possible to provide an imaging apparatus that can start" 
retrieval simultaneously with imaging. According to 

25 the invention, it is possible to provide an imaging 

apparatus that can distinguish a generic image from a 
key image easily, can use the key image repeatedly, and 
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can extract a key image suitable for the storage medium 
promptly. It is possible to provide an imaging 
apparatus that, even when a key image is not 
distinguished in a storage medium, for example, when a 
5 storage medium storing images imaged by other imaging 
apparatuses is used in the imaging apparatus, can 
specify a key image easily. 

It is possible to provide an imaging apparatus 
that can perform retrieval easily because, even if a 

10 retrieval operation is started in an image slightly 
distant from a desired key image, it becomes more 
likely that an image close to the desired key image 
appears while retrieval is repeated. It is possible to 
provide an imaging apparatus that can execute an 

15 imaging operation and a reproducing operation 

appropriately even when a retrieval operation continues 
in the background. It is possible to provide an 
imaging apparatus that allows a user to start retrieval 
with a touch of a retrieval button in a reproduction 

20 mode. It is possible to provide an imaging apparatus 
that makes it easy to find a key image. According to 
the invention, the imaging apparatus includes the 
operating unit that inputs a first operation input in 
acquiring generic image data and a second operation 

25 input in acquiring key image data in different 

operation forms. Thus, the user can distinguish two 
kinds of imaging operations from each other clearly and 
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select an image operation easily. 

According to this invention, in the imaging 
apparatus that stores generic image data and key image 
data on an identical storage medium, the imaging 
5 apparatus stores the generic image data and the key 
image data on the storage medium in distinguishable 
expressions. Thus, it is possible to select key image 
data more easily and promptly than in the past. 
According to the invention, the imaging apparatus 

10 executes retrieval processing using key image data with 
completion of imaging processing for the key image data, 
by the imaging unit as a trigger. Thus, the imaging 
apparatus can start retrieval after imaging a key image . 
According to the invention, the imaging apparatus can 

15 use images imaged by other imaging apparatuses as key 
image data. According to the invention, the imaging 
apparatus executes retrieval processing again on the 
basis of new key image data. Thus, it is possible to 
execute narrow-down retrieval and new retrieval with 

20 simple operation. 

According to the invention, access to a storage 
medium by the imaging unit and access to the storage 
medium by the retrieving apparatus are executed by 
different controls- Thus, even when retrieval 

25 processing is executed, the imaging unit can image 
generic image data in parallel with the retrieval 
processing. According to the invention, access to a 
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Storage medium by the reproduction processing unit and 
access to the storage medium by the retrieving unit are 
executed by different controls. Thus, even when 
retrieval processing is executed, it is possible to 
5 reproduce generic image data in parallel with the 

retrieval processing. According to the invention, when 
the retrieval processing is set in an operation mode, 
in which retrieval processing is executable, the key 
image data designated in advance is maintained as a 

10 retrieval key. Thus, it is possible to start retrieval 
with simple operation. 

Other features and advantages of the present 
invention will be apparent from the following 
description taken in conjunction with the accompanying 

15 drawings, in which like reference characters designate 
the same or similar parts throughout the figures 
thereof . 

BRIEF DESCRIPTION OF DRAWINGS 
20 The accompanying drawings, which are incorporated 

in and constitute a part of the specification, 
illustrate embodiments of the invention and, together 
with the description, serve to explain the principles 
of the invention. 
25 FIG. 1 is a block diagram showing an example of a 

structure of an imaging apparatus according to a first 
embodiment of the invention; 
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FIG. 2 is an external view of the imaging 
apparatus according to the first embodiment; 

FIG. 3 is a state transition diagram showing mode 
transition of the imaging apparatus according to the 
5 first embodiment; 

FIG. 4 is a flowchart showing an imaging operation 
and operations related to writing in a storage medium 
30 according to the first embodiment; 

FIG. 5 is a flowchart showing an imaging operation, 
10 retrieval processing, and the like according to the 
first embodiment; 

FIG. 6 is a flowchart showing an operation of a 
reproduction mode according to the first embodiment; 
FIG. 7 is a flowchart related to an imaging 
15 subroutine for a key image according to the first 
embodiment ; 

FIG. 8 is a flowchart of a retrieval subroutine 
according to the first embodiment; 

FIG. 9 is a flowchart of a key image selection 
20 subroutine according to the first embodiment; 

FIG. 10 is a block diagram showing an example of a 
structure of an imaging apparatus according to a second 
embodiment of the invention; 

FIG. 11 is a block diagram showing an example of a 
25 structure of an imaging apparatus according to a third 
embodiment of the invention; 

FIG. 12 is a block diagram showing an example of a 
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Structure of an imaging apparatus according to a fourth 
embodiment of the invention; 

FIG. 13 is a flowchart showing an operation of a 
reproduction mode according to the fourth embodiment; 
5 FIG. 14 is a flowchart showing a key image 

specifying subroutine according to the fourth 
embodiment ; 

FIG. 15 is a flowchart showing a retrieval 
subroutine according to a fifth embodiment of the 
10 invention; 

FIG. 16 is a flowchart showing an operation of a 
reproduction mode according to a sixth embodiment of 
the invention; 

FIG. 17 is a flowchart showing a retrieval 
15 subroutine according to the sixth embodiment; 

FIG. 18 is an external view of an imaging 
apparatus according to a modification; 

FIG. 19 is an external view of an imaging 
apparatus according to a modification; 
20 FIG. 20 is a block diagram showing an exemplary 

structure of an imaging apparatus according to an 
eighth embodiment of the invention; 

FIG. 21 is an exemplary flowchart of image 
retrieval processing according to the eighth 
2 5 embodiment ; 

FIG. 22 is a diagram showing an example of display 
of a retrieval result according to the eighth 
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embodiment ; 

FIG. 23 is another exemplary flowchart of the 
image retrieval processing according to the eighth 
embodiment ; 

5 FIG. 24 is still another exemplary flowchart of 

the image retrieval processing according to the eighth 
embodiment ; 

FIG. 25 is still another exemplary flowchart of 
the image retrieval processing according to the eighth 
1 0 embodiment ; 

FIG. 26 is an exemplary flowchart concerning 
display processing for a retrieval result according to 
the eighth embodiment; 

FIG. 2 7 is a diagram showing an example of display 
15 of a retrieval result according to the eighth 
embodiment ; 

FIG. 2 8 is a diagram showing an example of display 
of icons according to the eighth embodiment ; 

FIG, 2 9 is a diagram showing an example of a 
20 direct print system according to the eighth embodiment; 
and 

FIG. 3 0 is an exemplary flowchart of direct print 
processing according to the eighth embodiment. 



25 



BEST MODE FOR CARRYING OUT THE INVENTION 
In respective embodiments of the invention 
described below, an imaging apparatus having an image 
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retrieval function is mainly kept in mind. In image 
retrieval/ when a certain image is assumed to be a key- 
image/ other images related to this key image are 
extracted. As detailed methods for the image retrieval, 
5 other than the method of comparing images disclosed in 
the conventional technique and the like, there are 
techniques for comparing file names, acquisition time, 
comment information, and the like. Note that it is 
needless to mention that the invention does not depend 

10 on an image retrieval method and various image 
retrieval methods can be adopted. 
[First embodiment] 

FIG. 1 is a block diagram showing an example of a 
structure of an imaging apparatus according to a first 

15 embodiment of the invention. It is possible to connect 
another information apparatus 200 such as a PC or a 
printer to an imaging apparatus 100 in this example. 
FIG. 2 is an external view of the imaging apparatus 
according to the first embodiment. 

20 In FIGS. 1 and 2, reference numeral 100 denotes an 

imaging apparatus such as a digital camera, a digital 
video camera, a cellular phone with a camera, or a PDA 
with a camera. The imaging apparatus 100 can store a 
still image and a moving image as electronic data. The 

25 imaging apparatus 100 includes a shutter button 22A, a 
retrieval button 22B, a jog dial for mode setting 22C, 
a power switch 22D, display and file operation related 



•I 



I 



P205-0195WO 



buttons 22E, ZOOM & WIDE buttons 22F, and a DISP button 
22G. These buttons form an operating unit 22. 

A display unit 20 is formed by a liquid crystal 
display or the like. The display unit 20 displays a 
5 subject image, an image that has already been imaged, a 
key image in performing retrieval processing and a 
retrieval result, an operation guide for a user, and 
the like. A storage medium housing section, which is 
covered by a cap at the time of imaging, is provided on 
10 a side of the imaging apparatus 100. A storage medium 
3 0 is inserted in the storage medium housing section. 
As examples of the storage medium 30, there are a 
repeatedly readable and writable memory card such as a 
Compact Flash (registered trademark) card (a CF card) , 

r 

15 a magnetic storage medium, an optical storage medium, a 
magneto-optical storage medium, and the like. 

On the other hand, concerning an internal 
structure of the imaging apparatus 100, reference sign 
A in FIG. 1 denotes a subject image, reference numeral 

20 11 denotes an optical system such as a lens, 12 denotes 
a CCD (or a CMOS area sensor, etc.) serving as an 
imaging element, and 13 denotes an AD conversion unit. 
Reference numeral 14 denotes an image processing unit 
that applies signal processing and the like to RGB 

25 signals adapted to human sight. For example, the image 
processing unit 14 applies shading correction, a gamma 
conversion, and the like to a digital image input 
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signal inputted through the AD conversion unit 13. 
Reference numeral 15 denotes a RAM controller, which is 
connected to a main CPU bus 29 and is also connected to 
a RAM 16 serving as storing means. 
5 Reference numeral 17 denotes a (main) CPU that 

controls the entire imaging apparatus 100. Reference 
numeral 18 denotes a ROM that stores a program in which 
a control method and the like for the entire imaging 
apparatus are written. Reference numeral 19 denotes a 

10 ROMIF that plays a role of an interface between the ROM 
18 and a main CPU bus 29. Reference numeral 24 denotes 
a sound output unit such as a speaker or a buzzer. 
Reference numeral 25 denotes a sound output unit IF 
including a DA converter. Reference numeral 2 6 denotes 

15 a sound input unit such as a microphone. Reference 

numeral 27 denotes a sound input unit IF including an 
AD converter. 

Reference numeral 28 denotes a communication 
circuit serving as communicating means that should 

2 0 perform a function of communication with other 

information apparatus and the like. The communication 
circuit 28 executes operations complying with standards 
such as USB, LAN, wireless LAN, IrDA, Bluetooth, and 
IEEE1394. The communication circuit 28 communicates 

25 with the information apparatus 200 such as a PC or a 
printer via a communication unit 22 8 mounted on the 
information apparatus 200. Consequently, it is 



■t 



-f 



P205-0195WO 

- 19 - 

possible to transfer image data or the like imaged by 
the imaging apparatus 100 to the PC and store the image 
data or the like in the PC or directly print the image 
data or the like with the printer or the like. It is 
5 also possible to remotely operate the imaging apparatus 
100 from the information apparatus 200. 

The display unit 20 is connected to the main CPU 
bus 29 via a display unit IF 21. It is possible to 
drive the display unit 20 from the (main) CPU 17. 

10 Similarly, the operating unit 22 including the shutter 
button 22A, the retrieval button 22B, the jog dial for 
mode setting 22C, the power switch 22D/ the display and 
file operation related buttons 22E, the ZOOM & WIDE 
buttons 22F, and the DISP button 22G is connected to 

15 the main CPU bus 29 via an operating unit IF 23. As a 
result, the imaging apparatus 100 recognizes an 
instruction, which is inputted from the operating unit 
22, through the operating unit IF 23. 

As shown in FIG. 1, the storage medium 3 0 is 

20 connected to the main CPU bus 2 9 via a storage medium 
IF 31. It is also possible to divide an internal area 
of the storage medium 30 into two areas, namely, a 
generic image storing area 301 and a key image storing 
area 302. Note that various kinds of means for 

25 dividing the internal area into two are possible. For 
example, the internal area may be divided by creating 
separate folders (directories) in an initial sequence 
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or the like of the imaging apparatus or may be divided 
such that the respective areas can be accessed 
according to an address area. 

A retrieval engine 50 is a circuit that compares 
5 key image data and generic image data and retrieves 
similar image data. The retrieval engine 50 may be 
formed by an ASIC or the like having a direct memory 
access (DMA) function. It is also possible to form the 
retrieval engine 50 with a sub-CPU that is capable of 

10 operating in parallel with the CPU 17. 
(Operation in an imaging mode) 

FIG. 3 is a state transition diagram showing mode 
transition of the imaging apparatus according to the 
first embodiment. FIG. 4 is a flowchart showing an 

15 imaging operation, processing for writing in the 

storage medium 30, and the like according to the first 
embodiment. FIG. 5 is a flowchart showing an imaging 
operation, retrieval processing, and the like according 
to the first embodiment. First, an operation in an 

20 imaging mode will be explained with reference to these 
drawings . 

It is assumed that the jog dial for mode setting 
22C is set in the imaging mode and the power switch 22D 
is pressed to turn ON a power supply. In this case, 
25 after the power supply is turned ON, when considerable 
time elapses and the initial sequence or the like ends, 
the imaging apparatus 100 transitions to a state of the 
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imaging mode (Tl) referred to in FIG. 3 and comes into 
a standby state. 

When the shutter button 22A is pressed in the 
imaging apparatus 100 that is in the imaging mode (Tl) 
5 and the standby state, usual imaging processing is 

executed. In other words, a subject image is imaged by 
the imaging unit and stored in the storage medium 3 0 as 
a generic image. On the other hand, in this embodiment, 
it is possible to image and store a key image in this 

10 state. For example, when the shutter button 22A and 
the retrieval button 22B are pressed simultaneously, 
the imaging apparatus 10 0 can image and store a key 
image. By adopting such a constitution, it is possible 
to distinguish imaging of a generic image from imaging 

15 of a key image easily in the imaging mode. 

In step S400, the CPU 17 polls a state of the jog 
dial 22C of the operating unit IF 23 to judge whether 
the imaging mode has ended. If the imaging mode has 
ended, the CPU 17 ends the processing in this flowchart. 

20 On the other hand, if the imaging mode continues, the 
CPU 17 proceeds to step S401. 

In step S401, the CPU 17 judges whether at least 
the shutter button 22A is pressed. For example, when 
the shutter button 22A is pressed, a shutter button ON 

25 signal is generated in the operating unit IF 23. Thus, 
the CPU 17 can detect the ON signal by executing 
polling. On the other hand, when the shutter button 
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22A and the retrieval button 22B are pressed 
simultaneously, a key image imaging signal (a 
simultaneous button press signal) is generated in the 
operating unit IF 23 in addition to the shutter button 
5 ON signal. The generated shutter button ON signal is 
communicated to the (main) CPU 17 in both imaging of a 
generic image and imaging of a key image. Note that, 
in detecting the shutter button ON signal, the shutter 
button ON signal may be detected by interrupt 
10 processing instead of the polling operation. If the 

shutter button ON signal cannot be detected, the CPU 17 
returns to step S400. On the other hand if the shutter 
button ON signal is detected, the CPU 17 proceeds to 
step S402. 

15 In step S402, the CPU 17 starts an imaging 

operation. Specifically, the CPU 17 photoelectrically 
converts an image of a subject image A, which is made 
incident on the CCD 12 via the optical system 11, for 
each pixel and sends the image to the AD conversion 

20 unit 13. The AD conversion unit 13 subjects an analog 
value concerning light and shade of each color of each 
pixel to AD conversion and sends a digital image input 
signal to the image processing unit 14. Next, the 
image processing unit 14 executes signal processing 

25 such as shading correction and gamma conversion on the 
inputted digital image input signal to converts the 
digital image input signal to RGB signals adapted to 
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human sight. As a result, respective pixel data or the 
like of still image data subjected to image processing 
are written and stored in the RAM 16 one after another 
via the RAM controller 15. When all pixel data and 
5 additional data for one image are written in the RAM 16, 
the imaging operation for the one image ends. Note 
that, in this embodiment, the data are not written in 
the storage medium 3 0 immediately every time the 
imaging operation for one image ends and all the data 

10 are written in the storage medium 30 together. 

In step S403, the CPU 17 judges whether the image 
data stored in the RAM 16 is generic image data or key 
image data. In performing this judgment, the CPU 17 
accesses the operating unit IF 23 to thereby check 

15 whether the key image imaging signal is generated. As 
a result of the judgment, if the imaging operation is 
imaging of a generic image, the CPU 17 proceeds to step 
S404 (T3) . On the other hand, if the imaging operation 
is imaging of a key image, the CPU 17 proceeds to step 

20 S405 (T4) . 

In step S404, the CPU 17 writes the image written 
and stored in the RAM 16 in the storage medium 30 as a 
generic image. For example, the CPU 17 transfers the 
image stored in the RAM 16 to the generic image storing 

25 area 3 01 of the storage medium 30 and stores the image 
therein . 

In step S405, the CPU 17 writes the image written 
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and stored in the RAM 16 in the storage medium 30 as a 
key image. For example, the CPU 17 transfers the image 
stored in the RAM 16 to the key image storing area 3 02 
of the storage medium 3 0 and stores the image therein. 
5 As described above, operation of only the shutter 

button is used as an operation input when generic image 
data is acquired and, on the other hand, simultaneous 
pressing of the shutter button and the retrieval button 
is used as an operation input when key image data is 

10 acquired. Consequently the generic image data and the 
key image data are inputted to the CPU 17 in different 
operation forms. Therefore, a user can easily 
distinguish the two kinds of imaging operations from 
each other clearly and select an imaging operation. 

15 A generic image and a key image are stored in an 

identical storage medium and a storing area for the 
generic image and a storing area for the key image are 
separated. Consequently, it is possible to store key 
image data and generic image data in a storage medium 

20 in distinguishable expressions. In general, since the 
number of key image data is smaller than the number of 
generic image data, it is possible to extract and 
select a key image from the storage medium 30 easily 
and promptly. 

25 Since key image data is stored in the nonvolatile 

storage medium 30, it is possible to use the key image 
data repeatedly. 



P205-0195WO 

- 25 - 

Since key image data and generic image data are 
stored in different storing areas, it is possible to 
distinguish the key image data from the generic image 
data easily. This is very useful for users. 
5 In particular, in future, as a capacity of a 

storage medium increases, it is expected that the 
number of generic images stored in one storage medium 
increases rapidly. Under such circumstances, it is 
possible to extract a key image promptly. Thus, the 
10 technique provided by the invention is a very effective 
technique . 

Preferably, when a key image is imaged, before 
transferring the image written and stored in the RAM 16 
to the key image storing area 302 and storing the image 

15 therein, in step S405, the CPU 17 may compress or 

curtail image data and, then, transfer the image to the 
storage medium 3 0 and store the image therein. In this 
case, it is desirable to adopt a compression ratio 
higher than a compression ratio for a generic image. 

20 This is because a key image is not always required to 
have a quality as high as that of a generic image. 
Therefore, by applying compression processing and the 
like, it is possible to save a storage capacity 
necessary for storage of the key image. As a result, 

25 it is possible to store a large number of image data in 
the storage medium 30, This is very useful for users. 

Incidentally, as described above, when the shutter 
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button 22A and the retrieval button 22B are pressed 
simultaneously in the imaging mode (Tl) , simultaneously 
with imaging of a key image, the CPU 17 may start 
retrieval processing in the storage medium 3 0 in the 
5 imaging apparatus 100. 

An operation in this case will be hereinafter 
explained with reference to the flowchart in FIG. 5. 
Note that the steps explained in FIG. 4 are denoted by 
the identical reference signs and explanations of the 

10 steps are omitted. 

In step S403, when the CPU 17 judges that, for 
example, the shutter button 22A and the retrieval 
button 22B are pressed simultaneously and an imaging 
operation for a key image is instructed, the CPU 17 

15 proceeds to step S501. The simultaneous pressing of 
the shutter button 22A and the retrieval button 22B 
means start of image retrieval processing. Note that 
the image retrieval processing is started with 
completion of the imaging of the key image (i.e., the 

20 key image is usable for retrieval processing) as a 
trigger. 

In step S501, the CPU 17 judges whether other 
retrieval processing has already been started. Since 
retrieval processing has an extremely heavy load, if 
25 two or more retrieval processes are started in parallel, 
it is likely that imaging processing, reproduction 
processing, and the like are adversely affected. Thus, 
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it is prohibited to start plural retrieval processings 
in parallel. If other retrieval processing has already 
been started/ the CPU 17 proceeds to step S502 and 
displays a message meaning "under other retrieval" on 
5 the display unit 20 via the display unit IF 21. On the 
other hand, if other retrieval processing has not been 
started/ the CPU 17 proceeds to step S503. Note that 
steps S501 and S502 are optional and may be omitted. 
This is because, if processing abilities of the CPU 17 
10 and the retrieval engine 50 are high, it is also 
possible to carry out plural image retrievals in 

« 

parallel . 

In step S503/ the CPU 17 starts retrieval 
processing while keeping the state of the imaging mode 

15 (Tl) . In the retrieval processing/ the CPU 17 performs 
operations described below. When the retrieval 
processing is started, an under -retrieval flag is set 
in the retrieval engine 50. The retrieval engine 50 
reads out generic images in the storage medium 3 0 

20 sequentially according to a direct memory access (DMA) 
operation without the intervention of the CPU 17 and 
transfers the generic images to the RAM 16. In this 
case, the retrieval engine 50 may transfer the generic 
images to an area different from an area for storing 

25 key images. The retrieval engine 50 compares images 
read out by the DMA operation and an image, which is 
just imaged, sequentially. When there is a similar 
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image, the retrieval engine 50 stores the image in 
still another area in the RAM 16. When the retrieval 
engine 50 has finished comparing all the generic images 
in the storage medium 30 in this way, the retrieval 
5 engine 50 releases the under- retrieval flag and sets a 
retrieval result presence flag in the retrieval engine 
while holding the similar image, which is a retrieval 
result, stored in the another area in the RAM 16. The 
retrieval engine 50 sends a retrieval end interrupt to 

10 the CPU 17. Consequently, the series of retrieval 

processing ends. Note that the CPU 17 may display the 
retrieval result on the display unit 20 after the 
retrieval ends or may be displayed on the display 20 
when the imaging apparatus 100 enters a reproduction 

15 mode described later. In the latter case, the CPU 17 

reads out the retrieval result presence flag and, when 
it is judged that a retrieval result is present, 
displays an image of the retrieval result on the 
display unit 20. 

20 Note that, simultaneously with the start of the 

retrieval processing, the CPU 17 may transfer the key 
image, which has just been imaged, stored in the RAM 16 
to the key image storing area 3 02 of the storage medium 
30 and store the key image therein {S405) . However, 

25 the key image in this case may be deleted from the RAM 
16 without being written when the retrieval processing 
ends . 
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Note that/ when the start of the retrieval 
processing (S503) and the writing operation for the key- 
image data performed as required (S405) end, the CPU 17 
ends the series of imaging processing for retrieval 
5 image data without waiting for end of the retrieval 
processing by the retrieval engine 50. In this case, 
even if the CPU 17 comes into a state in which the CPU 
17 can execute the next imaging operation, the 
retrieval engine 5 0 may be continuing the retrieval 

10 processing in the background. Since the retrieval 
processing is being performed according to the DMA 
operation by the retrieval engine 50, the CPU 17 can 
execute the imaging processing for an image in parallel. 
In other words, a generic image can be imaged and a key 

15 image can also be imaged. However, the CPU 17 performs 
control such that imaging of a key image is rejected or 
prohibited until the retrieval end interruption occurs 
or the under- retrieval flag is released. Incidentally, 
the imaging apparatus can also shift to the 

20 reproduction mode (T2) during retrieval and perform a 
reproducing operation. This is because the CPU 17 is 
responsible for most of processing concerning the 
reproduction mode. 

On the other hand, the CPU 17 can display an 

25 indication that retrieval is being performed on the 
display unit 20 by referring to the under-retrieval 
flag during retrieval. Therefore, a user can recognize 
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that the retrieval processing is being executed in the 
background in the imaging mode. If the CPU 17 
recognizes occurrence of the retrieval end interrupt 
and release of the under- retrieval flag, the CPU 17 may 
5 display an indication that the retrieval ends on the 
display 20. In this way, the user can grasp whether it 
is possible to image a key image. This is because, 
when the retrieval ends, it is possible to image a key 
image. Note that, if the CPU 17 performs control such 

10 a key image in use is not overwritten by the retrieval 
engine 50, it is possible to image other key images 
even during retrieval . 

As explained above, according to this embodiment, 
when the shutter button and the retrieval button are 

15 pressed simultaneously, the imaging apparatus 10 0 
images key image data and starts image retrieval 
processing according to the key image data. Thus, it 
is possible not only to distinguish a generic image 
from a key image easily but also to start image 

20 retrieval immediately after imaging the key image. In 
other words, work for pressing the retrieval button 
after the imaging processing is made urmecessary and 
operation by a user can be reduced. 

Since the retrieval engine 50, which is operable 

25 in parallel with the CPU 17, is adopted, it is possible 
to execute an imaging operation and a reproducing 
operation appropriately while continuing retrieval 
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processing in the background. Thus, a very useful 
imaging apparatus is provided. 
(Operation in a reproduction mode) 

Next, it is assumed that the jog dial for mode 
5 setting 22C is set in a reproduction mode and the power 
switch 22D is pressed to turn on a power supply. In 
this case, after the power ON, when considerable time 
elapses and the initial sequence or the like ends, the 
imaging apparatus 100 transitions to a standby state in 

10 a state of a reproduction mode (T2) referred to in FIG. 
3 . Processing in the reproduction mode will be 
hereinafter explained with reference to FIGS. 6 to 9 . 
In particular, FIG. 6 is a flowchart showing operations 
in the reproduction mode according to the first 

15 embodiment. The flowchart is started when the imaging 
apparatus 100 transitions to the standby state in the 
reproduction mode (T2) . 

In step S601, the CPU 17 accesses the retrieval 
engine 50 and reads out the retrieval result presence 

20 flag stored in the retrieval engine. Note that access 
means transmission of an instruction, reception of a 
response, transmission and reception of data, signals, 
and the like, readout and writing of data, and the like. 
In step S602, the CPU 17 judges whether the read- 

25 out retrieval result presence flag is set to mean 
presence of a retrieval result. In other words, if 
this flag is asserted, this means that the retrieval 
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processing has been executed in the imaging mode and 
the retrieval processing has ended. Thus, the CPU 17 
proceeds to step S603 . 

In step S603, the CPU 17 reads out a retrieval 
5 result (similar image data) stored in a part of areas 
of the RAM 16 and displays the retrieval result on the 
display unit 20. In this way, a result -retrieved 
during the imaging mode is displayed immediately after 
the imaging apparatus transitions to the reproduction 

10 mode (T2) . In general, when the imaging apparatus 100 
transitions to the reproduction mode, since a user is 
often free, of pressure, it is preferable for the user 
to check the retrieval result. 

In step S604, the CPU 17 judges whether a button 

15 meaning end of the display of the retrieval result 

(e.g., the retrieval button) is pressed. In this case, 
the operating unit IF 23 generates a retrieval display 
end flag in association with the retrieval button 22B. 
The CPU 17 can detect a display instruction for the 

20 retrieval result by detecting the retrieval display end 
flag by polling or interrupt. In other words, the user 
presses the retrieval button again after checking the 
retrieval result, whereby the CPU 17 ends the display 
of the retrieval result and transitions to step S605. 

25 In step S605, the CPU 17 transitions to an 

operation of a usual reproduction mode. 

On the other hand, in step S602, as a result of 



I 



I 



P205-0195WO 

- 33 - 

reading out the retrieval result presence flag in the 
retrieval engine 50, if this flag is not asserted, the 
retrieval processing is not performed in the imaging 
mode or the retrieval processing has not ended. In 
5 this case, the CPU 17 proceeds to step S605 and 

displays a generic image on the display unit 20. As a 
method of display in this case, it is possible to adopt 
various methods such as a method of displaying only one 
latest image and a method of displaying plural 

10 thumbnail images. 

Although not shown on the flowchart, when the 
generic image is displayed on the display unit 20, the 
CPU 17 displays generic images stored in the storage 
medium 3 0 sequentially according to operation input of 

15 the display and file operation related buttons 22E such 
as an arrow key for inputting display operation and 
file operation. The CPU 17 may execute thumbnail 
display, zoom display of image data, and the like. 
Note that the thumbnail display and the zoom display 

20 are publicly- known techniques and can be carries out by 
those skilled art satisfactorily. Thus, further 
explanations of the thumbnail display and the zoom 
display are omitted. 

In step S606, the CPU 17 executes polling for 

25 detecting end of the reproduction mode in parallel with 
the display processing on the display unit 20. If the 
reproduction mode does not end, the CPU 17 proceeds to 
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Step S607 and repeats the polling for the retrieval 
result presence flag. If a retrieval .result is stored 
in the storage medium 30 or the like anew, the CPU 17 
proceeds to step S603 and displays the retrieval result. 
5 When the display operation ends, the CPU 17 negates the 
retrieval result presence flag of the retrieval engine 
50 (S604) . Thus, the CPU 17 always proceeds to step 
S608 until the reproduction mode is switched to the 
imaging mode next time. 

10 In this way, if a retrieval result is not present, 

the CPU 17 proceeds to step S608 and continues to 
polling pressing of the various buttons. As the 
various buttons, there are, for example, the shutter 
button 22A, the retrieval button 22B, and a key image 

15 selection button in the display and file operation 

related buttons 22E, and the like. It is needless to 
mention that pressing of these various buttons may be 
detected by interrupt processing instead of the polling. 
When the shutter button 22A (the operating unit 22) is 

20 pressed by the user, the operating unit IF 23 shown in 
FIG. 1 generates a shutter button ON signal. On the 
other hand, when the shutter button 22A and the 
retrieval button 22B are pressed simultaneously, the 
operating unit IF 23 generates a simultaneous pressing 

25 signal in addition to the shutter button ON signal. 

In the polling processing in step S606, when the 
CPU 17 detects that the jog dial for mode setting 22C 
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is other operation modes such as the imaging mode or 
the power switch 22D is pressed, the CPU 17 ends the 
reproduction mode. 

On the other hand, when the CPU 17 detects 
5 pressing of the shutter button 22A, the CPU 17 proceeds 
to step S609 and executes imaging processing for a key 
image (FIG. 7) . When the CPU 17 detects pressing of 
the retrieval button 22B, the CPU 17 proceeds to step 
S610 and executes retrieval processing (FIG. 8) . When 

10 the CPU 17 detects pressing of the key image selection 
button, the CPU 17 proceeds to step S611 and executes 
selection processing for a key image (FIG. 9) . 

(a) When the CPU 17 detects pressing of the 
shutter button 22A 

15 FIG, 7 is a flowchart showing a key image imaging 

subroutine according to the first embodiment. This 
flowchart corresponds to step S609 in FIG. 6. The 
flowchart is started, for example, when the CPU 17 
detects pressing of the shutter button 22A in the 

20 reproduction mode. In this case, the imaging apparatus 
100 images a key image for image retrieval rather than 
a generic image. 

In this way, when the CPU 17 detects that the 
shutter button is pressed in a state in which the jog 

25 dial serving as a mode selecting unit is set in the 
reproduction mode, the CPU 17 images key image data. 
In other words, when the shutter button is pressed in 
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the imaging mode as described above, an operation form 
is different from that at the time when generic image 
data is imaged. Therefore, it is possible to 
distinguish imaging of a generic image from imaging of 
5 a key image easily. There is also an advantage that it 
is possible to image a key image easily in the 
reproduction mode without switching the reproduction 
mode to the imaging mode. 

In step S701, the CPU 17 shifts to a key image 

10 imaging mode (T3) that is a transitional mode. In 

general, in the reproduction mode, since the imaging 
apparatus 10 0 is not in an imageable state in which the 
imaging apparatus 100 can capture an image data, it is 
necessary to prepare for imaging. For example, the CPU 

15 17 activates the CCD (or the CMOS area sensor, etc.) 12, 
the AD conversion unit 13, the image processing unit 14, 
and the like, changes the imaging apparatus 100 to the 
imageable state, and, in the case of a retracting lens, 
moves the lens to a position for imaging to bring the 

20 imaging apparatus 100 to a state in which preparation 
for imaging is completed. When the preparation for 
imaging is completed, the CPU 17 proceeds to an imaging 
operation for a key image. 

In step S702, the CPU 17 executes the imaging 

25 operation. The CPU 17 writes all pixel data and 

additional data for one image in the RAM 16 according 
to the same principle and operation as the imaging in 
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the imaging mode. Consequently, the imaging operation 
for a key image for one image ends. Note that, in this 
embodiment, as in the imaging mode, the data is not 
written in the storage medium 30 immediately every time 
5 the imaging operation for one image ends and all the 
data are written in the storage medium 3 0 together as 
described later. 

In step S703, the CPU 17 detects and judges 
whether the operating unit IF 23 has generated a 

10 simultaneous pressing signal. When the simultaneous 
pressing signal is generated, the shutter button 22A 
and the retrieval button 22B are pressed simultaneously. 
According to the embodiment of the invention, when both 
the buttons are pressed simultaneously in the 

15 reproduction mode, the CPU 17 performs imaging of a key 
image and storage of the key image in the storage 
medium 3 0 and starts retrieval processing of the 
retrieval engine 50. On the other hand, when only the 
shutter button is pressed, the CPU 17 only images a key 

20 image and stores the key image in the storage medium 30 
and does not start the retrieval processing. In other 
words, in the reproduction mode, it is possible to 
decide, according to whether both the buttons are 
pressed simultaneously, whether the retrieval 

25 processing is started immediately after a key image is 
imaged or a key image is only imaged and stored. Note 
that it is also possible that a key image is imaged and 
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Stored when both the buttons are pressed simultaneously 
and imaging and retrieval processing are started when 
only the shutter button is pressed. In short, although 
the shutter button is operated in common, operation 
5 forms only have to be different for the respective 
cases. Note that operation of the shutter button is 
not always essential and operation forms only have to 
be different for the respective cases such that, as a 
result, it is possible to distinguish imaging of a 

10 generic image from imaging of a key image clearly. 

When the simultaneous pressing signal is detected, 
the CPU 17 proceeds to step S704. When the 
simultaneous pressing signal is not detected, the CPU 
17 proceeds to step S712. 

15 In step S704, the CPU 17 accesses the under- 

retrieval flag and checks whether other retrieval 
processing still remains. At this point, it is likely 
that retrieval processing started before, for example, 
at the time of the imaging mode, is still under 

20 execution. Since it is not preferable to start two ore 
more retrieval processes, the CPU 17 checks this flag. 
However, if a retrieval load can be reduced by, for 
example, providing plural retrieval engines, plural 
retrieval processes may be operated in parallel. If 

25 the under-retrieval flag is asserted, since other 

retrieval processing is currently being executed, the 
CPU 17 proceeds to step S711. On the other hand, if 



•1 



P205-0195WO 

- 39 - 

the under-retrieval flag is not asserted, the CPU 17 
proceeds to step S705. 

In step S711, the CPU 17 displays an indication 
that other retrieval is being executed on the display 
5 unit 20 and, then, proceeds to step S712 without 
starting new retrieval processing. 

In step S705, the CPU 17 sends a retrieval 
instruction to the retrieval engine 50 to instruct the 
retrieval engine 50 to retrieval an image on the basis 
10 of the key image stored in the RAM 16. The retrieval 

engine 50 starts the retrieval processing in accordance 
with this instruction. 

In steps S706 and S712, the CPU 17 writes a latest 
key image stored in the RAM 16 in the key image storing 
15 area 302 of the storage medium 30. However, if the 

retrieval processing has been started in step S705, the 
CPU 17 may dispose of the key image used for the 
retrieval processing after the retrieval processing 
ends without storing the key image on the storage 
20 medium 30. 

In steps S707 and S713, the CPU 17 leaves the key 
image imaging mode (T6) and returns to the standby 
state of the reproduction mode (T2) . For example, when 
the optical system 11 adopts a retracting lens, the CPU 
25 17 controls the lens to move to a position for housing. 

Note that, when the retrieval processing is not 
started in the reproduction mode and step S713 ends. 
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the series of procedures end here and the CPU 17 leaves 
the subroutine. In other words, the CPU 17 displays a 
generic image on the display unit 20 (S605) and returns 
to the original polling state or the like (S606 to 
5 S608) . 

On the other hand, when the retrieval processing 
has been started, in step S708, the CPU 17 waits for 
occurrence of retrieval end interrupt and release of 
the under-retrieval flag according to polling. The CPU 

10 17 may display nothing on the display unit 20 or 

display a generic image or the like until the retrieval 
processing ends. In addition, the CPU 17 may display 
retrieved images sequentially or display the images 
with the thumbnail system. If it is judged that the 

15 retrieval processing ends, the CPU 17 proceeds to step 
S709 . 

In step S709, the CPU 17 displays all results on 
the display unit 20 sequentially or with the thumbnail 
system or the like. Consequently, the user can access 

20 all retrieval results. 

In step S710, the CPU 17 judges whether the 
imaging mode for a key image should be finished. For 
example, the CPU 17 checks whether the retrieval button 
22B is pressed again in the operating unit IF 23 and a 

25 retrieval button ON signal is generated. When the CPU 
17 detects the retrieval button ON signal meaning the 
end, the CPU 17 ends this subroutine and returns to the 
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main routine. In other words, the CPU 17 displays a 
generic image on the display unit 20 (S605) and returns 
to the original polling state or the like (S606 to 
S608) . 

5 (b) When pressing of the retrieval button 22B is 

detected 

In step S608 in FIG. 6, when the retrieval button 
22B is pressed during the reproduction mode, the 
retrieval processing is executed using a key image 

10 selected at this point. 

FIG. 8 is a flowchart of a retrieval subroutine 
according to the first embodiment. This subroutine is 
equivalent to step S610. 

In step S801, the CPU 17 accesses the under- 

15 retrieval flag and checks whether other retrieval 

processing is started. If the under-retrieval flag is 
asserted, this means that other retrieval processing is 
currently being executed. Thus, the CPU 17 proceeds to 
step S806, displays an indication on the display unit 

20 20 that other retrieval processing is being executed, 
and ends the subroutine without starting new retrieval 
processing. On the other hand, if the under-retrieval 
flag is not asserted, this means that other retrieval 
processing is not currently being executed. Thus, the 

25 CPU 17 proceeds to step S802. 

In step S802, the CPU 17 sends an instruction for 
starting retrieval processing to the retrieval engine 
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50. The retrieval engine 50 starts the retrieval 
processing on the basis of a key image that is 
currently set . Note that it is assumed that an 
arbitrary key image is always selected as a retrieval 
5 key and stored in a specific area of the RAM 16 by the 
CPU 17. 

There is an advantage that, for example, if a key 
image acquired latest is selected as a key image, a 
user can start retrieval with a touch of the retrieval 

10 button 22B in the reproduction mode. In addition, a 

key image highly likely to be used for retrieval can be 
set. On the other hand, a key image acquired earliest 
may be selected as a key image. In addition, a 
selected key image may be displayed in all or a part of 

15 the display unit 20. On the other hand, when the user 
desires to perform retrieval processing with a key 
image different from a key image selected by the 
imaging apparatus 100, it is possible to perform 
setting for change or the like of the key image by 

20 pressing the key image selection button described later 
in FIG. 9. 

In step S803, the CPU 17 waits for occurrence of 
retrieval end interrupt and release of the under- 
retrieval flag according to polling. The CPU 17 may 
25 display nothing on the display unit 20 or display a 
generic image or the like until the retrieval 
processing ends. Alternatively, the CPU 17 may display 
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retrieved images sequentially or display the images 
with the thumbnail system. If it is judged that the 
retrieval processing ends, the CPU 17 proceeds to step 
S804. 

5 In step S804, the CPU 17 displays all results on 

the display unit 2 0 sequentially or with the thumbnail 
system or the like. Consequently, the user can access 
all retrieval results. 

In step S805, the CPU 17 judges whether the 
10 retrieval processing should be finished. Since this 

judgment is the same as the judgment explained in step 
S710, an explanation of the judgment is omitted. 

(c) When pressing of the key image selection 
button is detected. 
15 When the CPU 17 detects that the key image 

selection button, which is one of the display and file 
operation related buttons 22E, is pressed in the 
reproduction mode, the CPU 17 starts change and 
selection processing for a key image to be used for 
20 retrieving. 

FIG. 9 is a flowchart of a key image selection 
subroutine according to the first embodiment. This 
subroutine is equivalent to step S611 described above. 
In step S901, the CPU 17 accesses the under- 
25 retrieval flag and checks whether other retrieval 
processing is started. At this point, it is likely 
that retrieval processing is still being executed. 
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However, the key image selected before is used in this 
retrieval processing. Since the started retrieval 
processing is affected if the key image is changed 
unreasonably, it is necessary to prohibit selection and 
5 change processing for the key image. Thus, in this 
step, the CPU 17 checks whether the retrieval 
processing started before is still being executed. If 
other retrieval processing is not started, the CPU 17 
proceeds to step S902 . On the other hand, if other 

10 processing is started, the CPU 17 proceeds to step S904, 
displays a warning indication such as "under other 
retrieval" or the like on the display unit 20 and 
leaves this subroutine. 

In step S902, the CPU 17 shifts to a key image set 

15 mode (T8) . First, the CPU 17 reads out a key image 
from the key image storing area 3 02 of the storage 
medium 30 and displays the key image. In reading out 
and displaying the key image data, for example, the CPU 
17 sorts image data in order of acquisition time or in 

20 order of file names and reads out and displays the 

image data in accordance with the sorted order. It is 
needless to mention that the CPU 17 may display the 
image data with the thumbnail system. Consequently, it 
becomes easy for a user to find a key image. 

25 In this key image set mode {T8) , the CPU 17 is in 

a state in which the CPU 17 is capable of accepting 
pressing of an arrow key (a triangle key) , which is one 
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of the display and file operation related buttons 22E, 
a decision button, which is one of the display and file 
operation related buttons 22E, and the like. Thus, the 
user can select one key image from the displayed images 
5 by using the arrow key (the triangle key) . In other 
words, the user can select one key image stored on the 
storage medium 30. When the CPU 17 detects pressing of 
the decision button at the point when the key image is 
selected, the CPU 17 changes and selects a key image 

10 that is used for retrieval. 

In step S903, the CPU 17 judges whether the key 
image set mode should be finished. For example, when 
the CPU 17 detects pressing of the key image selection 
button again, the CPU 17 ends this sxibroutine and 

15 returns to step S605, 

As described above, in the first embodiment, a 
first operation input in acquiring generic image data 
and a second operation input in acquiring key image 
data are inputted in different operation forms. Thus, 

20 the user can distinguish two kinds of imaging 

operations for a generic image and a key image from 
each other clearly. Since at least the shutter button 
22A is used in both the cases, the user can distinguish 
the two kinds of imaging operations from each other 

25 clearly while being conscious that imaging is performed. 
In addition, since the operation input in acquiring key 
image data can be performed by using the shutter button 
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22A together with other buttons, the user can image a 
key image while being conscious that imaging processing 
is perfoanned and with simple operation. Note that it 
is desirable that the shutter button 22A and the other 
5 buttons are arranged in positions allowing the user to 
press the buttons simultaneously . 

In the first embodiment, it is possible to acquire 
key image data even in a mode other than the imaging 
mode by at least operating the shutter button for input 

10 in the reproduction mode. 

In the first embodiment, it is possible to start 
retrieval immediately after imaging a key image by 
performing the second operation input in the imaging 
mode and the reproduction mode. Note that, when only 

15 the first operation input is performed in the 

reproduction mode, a key image is stored and retrieval 
is not performed. Thus, it is possible to distinguish 
a generic image and a key image with different plural 
operation inputs. 

20 It is possible to perform retrieval easily using 

an imaged image by making full use of the shutter 
button and the retrieval button according to the modes 
such as the imaging mode and the reproduction mode. In 
addition, the imaging apparatus stores a generic image 

25 and a key image in distinguishable expressions in an 

identical storage medium. Thus, even after imaging, it 
is possible to recognize the generic image and the key 
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image distinguishing one from another. Note that it is 
possible to use the key image repeatedly by storing the 
key image in a nonvolatile storage medium such as a 
flash memory. 

5 In this embodiment, it is possible to directly 

image an image that is used as a key image. As a 
result, since it is extremely easy to input the key 
image, it is possible to use a retrieval function 
conveniently . 

10 According to this embodiment, in addition to the 

CPU 17 that controls the imaging processing, the 
reproduction processing, and the like, the retrieval 
engine 50 operating in parallel with the CPU 17 is 
mounted on the imaging apparatus 100. Thus, there is 

15 an advantage that, even during execution of the 
retrieval processing, it is possible to continue 
imaging and reproduction of an image. 

According to the reproduction mode in this 
embodiment, arbitrary image data such as retrieved 

20 image data with the latest acquisition time and 

earliest retrieved image data is set as a retrieval key. 
Thus, the user can start retrieval processing simply by 
operating the retrieval button at arbitrary timing. 
[Second embodiment] 

25 In a second embodiment of the invention, the 

method of managing generic image data and key image 
data in the first embodiment is changed. 
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FIG. 10 is a block diagram showing an example of a 
structure of an imaging apparatus according to the 
second embodiment. Comparing with FIG. 1, it is seen 
that a method of storing image data on the storage 
5 medium 3 0 is different. In the first embodiment, the 
storage medium 30 includes the generic image storing 
area 301 and the key image storing area 302. On the 
other hand, in the second embodiment, both a generic 
image and a key image are stored in an identical area 

10 of the storage medium 30. Instead, different kinds of 
file management information are attached to a file 
allocation table (FAT) section of a generic image file 
and a FAT section of a key image file. In this 
embodiment, a data bit "0" is set in a reference 

15 portion of the FAT section for a generic image and a 
data bit "1" is set in a reference portion of the FAT 
section for a key image. Note that the file management 
information may be set in image information itself. 

In storing a key image on the storage medium 3 0 in 

20 step S405, S706, or S712, the CPU 17 stores the key 

image without specifically being conscious of a storing 
area. However, the CPU 17 stores the key image with 
file management information different from that of the 
generic image data attached in the FAT section, a file 

25 of the key image data, or the like. Consequently, it 
is possible to distinguish the generic image from the 
key image easily. 
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In designating a key image in step S902, the CPU 
17 can display candidates of the key image on the 
display unit 20 by reading out the image data attached 
with the file management information indicating the key 
5 information. 

As explained above, according to this embodiment, 
it is possible to obtain the same convenience as the 
first embodiment without distinguishing both the 
storing area from each other by storing file management 

10 information of key image data in the storage medium 3 0 
with identification information, which indicates that 
the key image data is different from generic image data, 
attached to the file management information . 
[Third embodiment] 

15 In the embodiments described above, a generic 

image and a key image are stored in the storage medium 
30 separately. 

However, if the key image cannot be used as the 
generic image, this is inconvenient. In particular, 

20 many problems occur when the key image is formed well. 
For example, it is conceivable to copy the key image 
and use the copy as generic image data. However, in 
this case, identical image data are stored redundantly 
and a storage capacity of a storage medium is wasted. 

25 Nevertheless, if an image to be used as a key image is 
found out of all generic images and set as in the past, 
operation is complicated. 
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Thus, in the third embodiment, although all images 
are treated as generic images, for a generic image that 
a user desires to use also as a key image, file 
management information different from that of the other 
5 generic images is attached in a FAT section or the 

inside of a file. Here, as in the second embodiment, a 
data bit "0" is set in a reference portion for a 
generic image and a data bit "1" is set in a reference 
portion for a key image. 

10 FIG. 11 is a block diagram showing an example of a 

structure of an imaging apparatus according to the 
third embodiment. In the third embodiment, an image 
treated as a key image (for special purpose) in the 
first and the second embodiments is treated as an image 

15 for both a key image and a generic image. 

In storing a key image on the storage medium 3 0 in 
step S404, S405, S706, or S712, the CPU 17 stores the 
key image with file management information meaning an 
image used as both a key image and a generic image 

20 attached in a FAT portion, a file of key image data, or 
the like. The file management information meaning an 
image used as both a key image and a generic image may 
be any information as long as the information can be 
distinguished from file management information 

25 dedicated for a generic image and file management 
information dedicated for key image. 

Note that the CPU 17 may store the image without 



»i P205-0195WO 



- 51 - 



specifically being conscious of a storing area or may 
store the image in the generic image storing area 301 
and the key image storing area 302 separately. In 
other words, it is possible to treat the key image data 
5 stored in the key image storing area 3 02 as the image 
used as both a generic image and a key image or treat 
the generic image data stored in the generic image 
storing area 301 as the image used as both a generic 
image and a key image. This is because it is possible 

10 to distinguish the image used as both a generic image 
and a key image according to the file management 
information in both the cases. 

In designating a key image in step S902, the CPU 
17 can display candidates of the key image on the 

15 display unit 20 by reading out image data attached with 
the file management information indicating the image 
used as both a generic information and a key 
information from the storage medium 30. In this way, 
it is possible to obtain the same convenience as the 

20 first embodiment. This is very useful for users. 

As explained above, in the third embodiment, 
info2nnation meaning the image used as both a generic 
image and a key image is attached to image data of a 
generic image used also as a key image or image data of 

25 a key image used also as a generic image. This makes 
it unnecessary to store both the generic image and the 
key image redundantly. It is possible to make use of a 
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Storage capacity of a storage medium effectively. 
Moreover^ there is also an advantage that a user can 
find a key image easily. 
[Fourth embodiment] 
5 In the first, the second, and the third 

embodiments, in writing image data in the storage 
medium 30, a key image and a generic image are stored 
in distinguishable expressions. However, when the 
storage medium 3 0 is detachably replaceable, image data 

10 imaged by other imaging apparatuses may be stored. 

Moreover, when such a storage medium 3 0 is used in the 
imaging apparatus 100, a key image and a generic image 
in the storage medium 30 cannot be distinguished from 
each other in some cases . 

15 In a fourth embodiment of the invention, an 

imaging apparatus that is useful even in such a case 
will be explained. 

FIG. 12 is a block diagram showing the imaging 
apparatus according to the fourth embodiment. In this 

20 example, for convenience of explanation, it is assumed 
that all image data are stored on the storage medium 30 
in an identical state. Note that the imaging apparatus 
100 shown in FIG. 12 does not have to have the 
respective functions explained in the first embodiment. 

25 In addition, the imaging apparatus in that case does 
not have to store a key image and a generic image on 
the storage medium 3 0 separately or may store the 
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images separately, 

FIG. 13 is a flowchart showing an operation of a 
reproduction mode according to the fourth embodiment. 
The steps explained in FIG. 6 are denoted by the 
5 identical reference signs and explanations of the steps 
are omitted. Note that, although it is assumed that an 
operation or the like in pressing a shutter button in 
the reproduction mode is identical with the operation 
shown in FIG. S, the operations may be different. 

10 According to FIG. 13, when a key image specifying 

button is detected, the CPU 17 proceeds to step S1301 
and executes key image specifying processing. It is 
assumed that the key image specifying button is also 
included in the operating unit 22. 

15 In order to use image data written by other 

imaging apparatuses or the like as a key image, it is 
necessary to specify the key image. 

This key image specifying button is different from 
the key image selecting button described above, which 

2 0 targets only a key image, in that the imaging apparatus 
enters a mode for finding an image allocated as a key 
image out of all images. One button of the display and 
file operation related buttons 22E may be allocated 
anew as this key image specifying button, 

25 Alternatively, the key image specifying button may be 
allocated instead of the key image selecting button. 
FIG. 14 is a flowchart showing a key image 
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specifying subroutine according to the fourth 
embodiment. The subroutine is equivalent to step S1301 
in FIG. 13. Note that the steps explained in FIG. 9 
are denoted by the identical reference signs and 
5 explanations of the steps are omitted. 

In step S901, the CPU 17 checks whether other 
retrieval processing is started. In step S902, the CPU 
17 reads out image data stored on the storage medium 3 0 
and displays the image data on the display unit 20 as 

10 candidates. When any one of the image data is 

specified by the operating unit 22, the CPU 17 proceeds 
to step S1401. Here, a key image is specified out of 
all images, which is different from the change and the 
selection of a key image described above. 

15 In step S1401, the CPU 17 judges whether the image 

data specified by the operating unit 20 should be 
stored as a key image according to a state of the key 
image set mode or after the specifying processing ends. 
For example, the CPU 17 detects pressing of a storing 

20 button, to which one of the display and file operation 
related buttons 22E is allocated, to judge that the 
image data should be stored in the storage medium 3 0 as 
a key image. When the image data should be stored as a 
key image, the CPU 17 proceeds to step S1402 . 

25 In step S1402, the CPU 17 stores the specified 

image data in the storage medium 3 0 as a key image. 
In this way, it is possible to specify one of 
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image data, which are simply stored in the storage 
medium 30/ and store the image data by overwriting or 
store the image data anew as a key image. In this case, 
it is also possible that the key image storing area 302 
5 is created to store the image data therein or the image 
data is stored with file management information 
indicating a key image or an image used as both a 
generic image and a key image attached thereto. 

Thereafter, when the CPU 17 judges in step S903 

10 that the specifying processing should be finished, the 
CPU 17 returns to the main routine. 

According to the fourth embodiment, even if it is 
impossible to find whether the image data stored in the 
storage medium 3 0 is a key image or a generic image, it 

15 is possible to specify a key image easily. For example, 
this is effective when the storage medium 3 0 including 
image data imaged by other imaging apparatuses or the 
like is used in the imaging apparatus 100. 

In addition, the image data specified as a key 

20 image is stored on the storage medium 30 as a key image. 
Thus, it is possible to use the key image meaningfully 
in the imaging apparatus 100 according to the first, 
the second, or the third embodiment. 
[Fifth embodiment] 

25 In a fifth embodiment of the invention, the 

retrieval subroutine shown in FIG. 8 is improved. 
Depending on convenience of a user, for example, it is 
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conceivable to select an image slightly far from a 
desired key image and start retrieval processing. In 
that case, it is likely that the desired key image or 
an image closer to the key image is retrieved during 
5 the retrieval. In that case, it is considered that 
retrieval noise is reduced and a more sophisticated 
retrieval result is obtained when the retrieved image 
closer to the desired key image is set as a key image 
to retrieve an image. Thus, in this embodiment, it is 

10 made possible to select image data found during 

retrieval processing as a retrieval key and execute the 
retrieval processing again. 

FIG. 15 is a flowchart showing a retrieval 
subroutine according to the fifth embodiment. The 

15 subroutine is equivalent to step S610 in FIGS. 6 and 13. 
Note that the steps explained in FIG. 8 are denoted by 
the identical reference signs and explanations of the 
steps are omitted. In the following explanation, it is 
assumed that, in step S804, image data, which a user 

20 desires to designate as a key image anew, is retrieved 
and displayed on the display unit 20. 

In step S1501, the CPU 17 judges whether a new key 
image is designated from a retrieval result. For 
example, when the CPU 17 detects that the retrieval 

25 button 22B is pressed again after an image appearing 

from the retrieval result is designated by an arrow key 
(a triangle key) or the like, the CPU 17 selects the 
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designated image data as a key image again, proceeds to 
step S802, and starts image retrieval using the key- 
image . 

On the other hand, if the designated image data is 
5 not selected as a key image again, the CPU 17 proceeds 
to step S1502 and judges whether the retrieval result 
is stored as a key image. For example, when the CPU 17 
detects that an image as the retrieval result or the 
key image used in the retrieval is designated by the 

10 arrow key (the triangle key) and the storing button is 
pressed, the CPU 17 executes judgment processing. When 
it is judged that the retrieval result should be stored 
as a key image, the CPU 17 proceeds to step S1503. 
Otherwise, the CPU 17 proceeds to step S805. 

15 In step S1503, the CPU 17 stores the designated 

image data in the storage medium 30. A method of 
storing a key image is as already explained. 

As explained above, according to this embodiment, 
it is possible to designate a new key image out of 

20 retrieved image data and execute retrieval processing 
again. Thus, there is an advantage that it is possible 
to execute narrow-down retrieval and new retrieval with 
simple operation. For example, since the number of 
image data stored in the storage medium 3 0 is enormous, 

25 it is possible to start retrieval processing with an 

image slightly far from a desired key image first and, 
when closer image data appears, designate the image as 
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a new key image, and execute the retrieval again . 
While the retrieval is repeated in this way, a 
probability of retrieval of images close to the desired 
key image is improved. 
5 [Sixth embodiment] 

In a sixth embodiment of the invention, a 
technique for converting a key image and a generic 
image mutually will be explained. 

FIG. 16 is a flowchart showing an operation of a 

10 reproduction mode according to the sixth embodiment. 

Note that the steps explained in FIG. 6 are denoted by 
the identical reference signs and explanations of the 
steps are omitted. 

First, it is assumed that, in step S605, a 

15 specific generic image is designated by an arrow key (a 
triangle key) when generic images are displayed on the 
display unit 20. 

In step S1601, the CPU 17 detects pressing of a 
change button to which one of the display and file 

20 operation related buttons 22E is allocated. When the 
pressing of the change button is detected, the CPU 17 
proceeds to step S1602, changes the designated generic 
image to a key image, and stores the key image on the 
storage medium 30. There are various methods of 

25 realizing the change processing. For example, the CPU 
17 may copy a generic image to create a key image. The 
CPU 17 may change file management information of a 
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generic image to file management information indicating 
a key image. The CPU 17 may create a link of generic 
image data and store the link in the key image storing 
area 302. Alternatively, the CPU 17 may move image 
5 data from the generic image storing area 301 to the key 
image string area 302. It is needless to mention that 
the image data may be managed as the image used as both 
a generic image and a key image. 

Note that it is needless to mention that steps 
10 S1601 and S1602 may be inserted in the flowchart of FIG. 
13 . 

FIG. 17 is a flowchart showing a retrieval 
subroutine according to the sixth embodiment. Note 
that the steps explained in FIG. 9 are denoted by the 
15 identical reference signs and explanations of the steps 
are omitted. 

When a key image is selected in step S902, if the 
change button is pressed in step S1701, the CPU 17 
proceeds to step S1702 and changes the selected key 

20 image to a generic image. A specific example of the 
change processing is the same as the processing 
explained about step S1602 except that an object of 
conversion is opposite. 

Note that it is needless to mention that steps 

25 S1701 and S1702 may be inserted in the flowchart of FIG. 
15 . 

As explained above, according to the sixth 
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erabodiment, it is possible to convert a key image and a 
generic image mutually. For example, when an image 
imaged as a key image is an image formed well, it is 
possible to use the image as a generic image. On the 
5 other hand, when it is expected that an image imaged as 
a generic image is also used as a key image, it is 
possible to use the generic image also as a key image. 
[Seventh embodiment ] 

The various embodiments have been explained above. 

10 However, it is needless to mention that a flowchart 

other than those mentioned above may be used in a range 
not departing from the technical thought of the 
invention or the variations described above may be 
combined in a range not causing contradiction. It is 

15 needless to mention that even such modifications belong 
to the technical scope of the invention as long as the 
modifications are substantially identical with the 
constitution of the invention described in claims. 

FIGS. 18 and 19 are external views of an imaging 

20 apparatus according to a modification. According to 
FIG. 18, the retrieval button 22B is arranged in a 
position of a left index finger. According to FIG. 19, 
the retrieval button 22B is arranged in a position of a 
left thumb. In short, the retrieval button 22B may be 

25 arranged in any position as long as a user can easily 
operate the retrieval button 22B simultaneously with 
the shutter button 22A. For example, it is possible to 
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arrange the retrieval button 22B in any position such 
as a front surface, a rear surface, a left surface, a 
right surface, an upper surface, and a lower surface of 
the imaging apparatus 100. 
5 In the embodiments described above, an operation 

such as imaging of a key image is started by pressing 
the retrieval button and the shutter button 
simultaneously. However, a user interface (UI) for 
instructing the start of the operation is not limited 

10 to this, A different form of UI may be adopted as long 
as both a retrieval button ON signal and a shutter 
button ON signal are detected in a camera. For example, 
a user may press the shutter button while pressing the 
retrieval button or may press the shutter button within 

15 a predeteirmined time after pressing the retrieval 
button . 

In the invention, in some parts of the explanation, 
the retrieval processing is executed in the 
reproduction mode. However, a retrieval mode intended 

20 mainly to be used for retrieval may be provided 

separately. In that case, it is needless to mention 
that the constitution of the invention can be applied. 
[Eighth embodiment] 

Incidentally, it is likely that a problem occurs 

25 in terms of visual recognition unless a screen size of 
a display device, in which similar image data as a 
retrieval result is displayed, is taken into account 
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sufficiently. For example, in a digital camera or a 
cellular phone with a camera function, even if plural 
similar image data are found, it is difficult to 
visually compare the similar image data to find which 
5 of the similar image data is similar to key image data 
due to the small size of the screen. 

In the invention described in Japanese Patent 
Application Laid-Open No. 2000-76302, retrieved four 
similar image data are arranged and displayed in an 
10 identical size. However, the similar image data and a 
key image data are not displayed simultaneously. 
Therefore, it is difficult to visually judge which of 
the similar image data is most similar to the key image 
data. 

15 Thus, an eighth embodiment of the invention, it is 

an object to solve such a problem and other problems. 
Note that the other problems will be understood through 
the entire specification. 

In order to solve the problems, the invention 

20 provides an imaging apparatus having a storage medium 

that stores plural retrieval object image data, wherein 
the imaging apparatus retrieves similar image data 
similar to key image data out of the plural retrieval 
object image data and displays the retrieved similar 

25 image data and the key image data in expressions 
visually distinguishable from each other. 

According to this embodiment, since the retrieved 
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similar image data and the key image data are displayed 
in expressions distinguishable from each other, a user 
can easily decide which similar image data is most 
similar to the key image data. 
5 <Example of a configuration of hardware> 

FIG. 20 is a block diagram showing an exemplary 
structure of an imaging apparatus according to this 
embodiment. The imaging apparatus 100 is an apparatus 
having an imaging function such as a digital camera, a 

10 digital video camera, a cellular phone with a camera, 
or a PDA with a camera. 

The imaging apparatus 100 converts reflected light 
of a subject made incident through a lens 101 into an 
electric signal with an imaging unit 102. The imaging 

15 unit 102 includes a photoelectric conversion device 
such as a charge coupled device (CCD) and a 
complementary metal oxide semiconductor (CMOS) sensor . 

The imaging unit 102 determines parameters 
necessary for photographing and displays the parameters 

20 on a display unit 109. As the parameters necessary for 
photographing, for example, there are information 
related to exposure (e.g., shutter speed and a value of 
stop), information indicating a focus state, and the 
like. These kinds of information are generally 

25 displayed on an internal liquid crystal display of an 
optical finder or an external liquid crystal display. 
Note that it is assumed that the display unit 10 9 in 
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this example is the external liquid crystal display. 

The imaging unit 102 acquires an electric signal 
corresponding to an image of a subject and outputs the 
electric signal to an image processing unit 103. The 
5 image processing unit 103 applies image processing for 
making it possible to store the electric signal as 
image data to the electric signal. The processing is 
controlled by a CPU 107 that operates in accordance 
with a program. Note that it is assumed that a code of 

10 a program is stored in a ROM 113 that is controlled by 
a ROM controller 108. 

A RAM 105, which is controlled by the RAM 
controller 104, stores data related to an operation of 
the program temporarily, 

15 A communication unit 106 is a communication 

circuit for communicating with an external device (e.g., 
a printer) . The communication unit 106 executes 
transmission processing or reception processing for 
data between the imaging apparatus 100 and the external 

20 apparatus such as a PC or a printer according to the 
control of the CPU 107. 

The image processing unit 103 executes various 
image processings on the electric signal outputted from 
the imaging unit 102 and generates image data. The 

25 image data subjected to the image processing is stored 
on a storage medium 114 such as a compact flash 
(registered trademark) card or a hard disk drive while 
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being controlled by a storage controller 111. 

When a shutter button provided in the operating 
unit 110 is operated, the CPU 107 executes 
photographing processing. In this example, it is 
5 assumed that there are two stages in a pressing amount 
of the shutter button. When the CPU 107 detects that 
the shutter button is pressed down to a first stage, 
the CPU 107 outputs an instruction to determine 
photographing parameters such as stop, shutter speed, 

10 and focus state to the imaging unit 102. When the CPU 
107 detects that the shutter button is pressed down to 
a second stage, the CPU 107 executes photographing 
processing for acquiring image data. Note that a 
mechanism of the shutter button is generally known 

15 regardless of whether the imaging apparatus 100 is a 
silver salt camera or a digital camera. Thus, further 
explanations about the shutter button would be 
unnecessary . 

A similar-image-retrieving unit 112 retrieves 

20 image data similar to key image data out of image data, 
which are stored in the storage medium 114 and the RAM 
105, as required. For example, when a user 
photographed a photograph of scenery (image data) 
during travel, the user retrieves a similar photograph 

25 of scenery among photographs photographed before. When 
the user photographed a person, the user retrieves a 
photograph of the person out of photographs in the past 
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or retrieves a photograph of a similar facial 
expression out of photographs in the past paying 
attention to facial expressions of the person. 

There are several most convenient examples of the 
5 method of retrieving a similar image. In a first 

method, for respective pixels in an image, gray scales 
of the pixels are binarized according to whether the 
scales are higher or lower than a threshold value to 
generate a binarized image. The generated binarized 

10 image and key image data are compared (e.g., similarity 
is obtained) to retrieve similar image data. As 
another method, there are a method of using a histogram 
representing brightness of an entire image and a method 
of performing contour extraction paying attention to a 

15 frequency component of an image. As another method, 
there is a method of comparing information such as 
color information, a shape of a subject, or inclination 
and performing similarity retrieval. Note that, in 
order to improve detection accuracy, similarity 

20 retrieval processing only has to be performed using a 
higher characteristic amount such as identifying a 
person and recognizing a facial expression of the 
person. It is needless to mention that the similarity 
judging methods are only examples and the invention can 

25 adopt other similarity judging method. 
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< Image retrieval processing> 

1. Image retrieval at the time of preview 
FIG. 21 is an exemplary flowchart of image 
retrieval processing according to this embodiment. In 
5 this image retrieval processing, in displaying preview 
image data before photographing processing, image data 
similar to the preview image data is retrieved. 

In step S2101, the CPU 107 detects whether the 
shutter button included in the operating unit 110 is 
10 pressed down to a first stage. If the shutter button 
is pressed down to the first stage, the CPU 107 
proceeds to step S2102. 

In step S2102, the CPU 107 controls the imaging 
unit 102 to determine photographing parameters. 
15 In step S2103, the CPU 107 generates preview image 

data complying with the determined parameters . 

In step S2104, the CPU 107 causes the similar- 
image- retrieving unit 112 to execute retrieval 
processing with the generated preview image data as key 
20 image data. The similar-image-retrieving unit 112 

retrieves similar image data similar to the key image 
data from the storage medium 114. 

In step S2105, the CPU 107 displays the retrieved 
similar image data and the preview image data in 
25 expressions visually distinguishable from each other. 

The CPU 107 may use any method to display the retrieved 
similar image data and the preview image data as long 
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as both the image data can be distinguished visually. 
For example, the retrieved similar image data and the 
preview image data are displayed in parallel. 

FIG. 22 is a diagram showing an example of display 
5 of a retrieval result according to this embodiment. 
Key image data (e.g., preview image data) 2200 and 
similar image data 2201 to 2203, which are retrieval 
results, are displayed on the display unit 109. 

In step S2106, the CPU 107 detects whether the 
10 shutter button included in the operating unit 110 is 
pressed down to the first stage. If a state in which 
the first stage of the shutter button is ON is not 
continued, the CPU 107 regards that photographing is 
stopped and returns to step S2101. If the ON state is 
15 continued, the CPU 107 proceeds to step S2107. 

In step S2107, the CPU 107 detects whether the 
shutter button included in the operating unit 110 is 
pressed down to the second stage. If the shutter 
button is pressed down to the second stage, the CPU 107 
20 proceeds to step S2108. 

In step S2108, the CPU 107 executes photographing 
processing. For example, the CPU 107 applies 
compression processing or the like to data outputted 
from the imaging unit 102 in the image processing unit 
25 103, creates image data, and writes the created image 
data in the storage medium 114. 

In step S2109, the CPU 107 displays the image data 
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subjected to the photographing processing on the 
display unit 109. 

According to this embodiment, image data similar 
to preview image data is retrieved out of image data 
5 stored in the storage medium 114 or the RAM 105 and the 
preview image data and the similar image data are 
displayed in expressions visually distinguishable from 
each other. Therefore, a user can easily evaluate the 
retrieved image data. 

10 According to this embodiment, since preview image 

data is used as key image data, it is possible to 
reduce useless photographing processing. For example / 
when the user intends to execute photographing for fear 
of forgetting photographing of a subject, if similar 

15 image data for the subject is present in a storage 

medium, the user can stop photographing. Therefore, it 
is possible to prevent useless photographing in advance. 

Conversely, when the user displays a subject, 
which the user forgot to photograph, as preview, since 

20 similar image data is not found and is not displayed on 
the display unit 109, the user can recognize that the 
user forgot to photograph the subject. Thus, an effect 
of preventing the user from forgetting photographing of 
an image can also be expected. Note that, if the CPU 

25 107 displays a warning such as "similar image data is 
not found" on the display unit 109 or a liquid crystal 
display section in a finder, it is easier to prevent 
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the user from forgetting photographing. 

Incidentally, in a RAW image or the like that 
takes time in photographing processing because of a 
large amount of data, the effect of stopping 
5 photographing processing is higher compared with JPEG 
or the like. Since image data of a RAW format has a 
large size of image data compared with the JPEG format 
or the like, a time lag from first photographing until 
the next photographing is large. When photographing is 

10 performed in such a RAW format, a user can easily- 
dee ide whether photographing is necessary on the basis 
of a retrieval result by retrieving similar image data 
in preview before photographing- According to this 
embodiment, since it becomes easy to avoid useless 

15 photographing processing in this way, it is possible to 
execute photographing processing smoothly even if image 
data of the RAW format is used. 

Note that, in executing image retrieval for the 
preview image data, the CPU 107 may execute the image 

20 retrieval after judging whether setting is made to 

execute the image retrieval. For example, the CPU 107 
makes setting in advance to execute or skip image 
retrieval at the time of preview according to an 
instruction from the operating unit 110. The CPU 107 

25 stores contents of setting in the ROM 113 such as an 

EEPROM and uses the contents of setting as a reference 
at the time of judgment. If setting is made not to 
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execute image retrieval at the time of preview 
depending on convenience of a user, since photographing 
is never interrupted by the image retrieval, the user 
can concentrate on the photographing . 
5 2 . Image retrieval immediately after photographing 

In the embodiments described above, a similar 
image is retrieved at the time of preview processing. 
However, in this embodiment, image retrieval is carried 
out immediately after photographing processing. 

10 FIG. 23 is another exemplary flowchart of the 

image retrieval processing according to this embodiment. 
Note that the steps already explained are denoted by 
the identical reference signs to simplify explanations. 
After executing steps S2101 to S2103, in step 

15 S23 04, the CPU 107 displays the preview image data on 
the display unit 109. Thereafter, the CPU 107 executes 
steps S2106 to S2108, executes photographing processing 
for a subject, and proceeds to step S2309. 

In step S2309, the CPU 107 causes the similar- 

20 image- retrieving unit 112 to execute retrieval 
processing with the image data subjected to the 
photographing processing as key image data. The 
similar- image-retrieving unit 112 retrieves similar 
image data similar to the key image data from the 

25 storage medium 114 or the RAM 105. 

In step S2310, the CPU 107 displays the retrieved 
similar image data and the image data of the subject in 
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expressions visually distinguishable from each other. 
For example, as shown in FIG. 22, the CPU 107 displays 
the retrieved similar image data and the image data of 
the subject in parallel. 
5 According to this embodiment, immediately after 

the photographing processing and before the next 
photographing processing, a user can compare and 
evaluate image data obtained by the photographing 
processing and existing image data. For example, when 

10 photographing is redone for a certain subject, it 

becomes easy to evaluate a first photographing result 
and a photographing result obtained by redoing the 
photographing . 

Note that, as described above, the CPU 107 may 

15 execute image retrieval after judging whether setting 
is made to execute the image retrieval. The user can 
select whether the image retrieval should be executed 
at the time of preview, should be executed immediately 
after photographing, or should not be executed at all 

20 at these timings. Note that the image retrieval 

processing may be executed at the time of inspection of 
image data. For example, the CPU 10 7 displays image 
data selected by the operating unit 110 on the display 
unit 109 and, when image retrieval is instructed by the 

25 operating unit 110 subsequently, executes the image 

retrieval and displays similar image data. This makes 
it possible to execute image retrieval at the time of 
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inspection of image data and display key image data and 
similar image data in forms that can be compared with 
each other. 

3. Skip of image retrieval processing at the time 
5 of continuous photographing 

Image retrieval processing relatively has a heavy 
load compared with processing for photographing one 
image data. Therefore, when the image retrieval 
processing is executed for each of image data obtained 
10 at the time of continuous photographing, it is likely 
that continuous photographing speed falls excessively. 
Thus, in this embodiment, when continuous photographing 
processing is executed, the fall in continuous 
photographing speed is controlled by skipping retrieval 
15 processing for similar image data. 

FIG. 24 is still another exemplary flowchart of 
the image retrieval processing according to this 
embodiment. Note that the steps already explained are 
denoted by the identical reference signs to simplify 
20 explanations. 

After executing steps S2101 to S2107, in step 
S2108, the CPU 107 applies photographing processing to 
one image data. Thereafter, the CPU 107 proceeds to 
step S2401. 

25 In step S2401, the CPU 107 judges whether 

continuous photographing is instructed by the shutter 
button of the operating unit 110. For example, the CPU 
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107 judges whether the shutter button is continuously- 
pressed down to the second stage. When the continuous 
photographing is instructed, the CPU 107 skips 
retrieval processing for similar image data and 
5 proceeds to step S2401. On the other hand, when the 

continuous photographing is not instructed, the CPU 107 
proceeds to step S2309. In step S2309, the CPU 107 
executes image retrieval on image data photographed 
last. Note that the CPU 107 may execute the image 

10 retrieval on all image data, which are continuously 

photographed, sequentially. Alternatively, the CPU 107 
may select a part of the image data and execute image 
retrieval processing on the image data according to an 
operation instruction from the operating unit 110. 

15 As explained above, according to this embodiment, 

the CPU 107 perfoirms control to stop retrieval 
processing while the continuous photographing 
processing is instructed by the shutter button. Thus, 
it is possible to control the fall in continuous 

20 photographing performance. 

Note that, since similar images are photographed 
continuously at the time of continuous photographing, 
it is useless to execute image retrieval processing on 
all the similar images. Therefore, it is possible to 

25 execute similar image retrieval efficiently by- 
executing the image retrieval processing on only image 
data acquired last and selected image data. 
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4. Prioritization of image retrieval objects 
When a large amount of image data are stored in 
the storage medium 114, processing time for image 
retrieval increases- However, necessity of executing 
5 image retrieval on all stored image data is not always 
high. For example, when plural similar image data are 
present, it seems that data with new photographing time 
is closer to key image data than data with old 
photographing time. When a user desires to retrieve an 

10 image paying attention to image data that was 

photographed so long time ago that the user does not 
remember, it is advisable to perform retrieval in order 
form image data with old photographing time. 

When attention is paid to an amount of data of 

15 image data, it is considered that the image data is 
more likely to be similar to key image data as the 
amount of data is closer to an amount of the key image 
data. In addition, when attention is paid to 
photographing parameters, it is considered that image 

20 data is more similar to key image data as photographing 
parameter of the image data are closer to photographing 
parameters of the key image data . 

In any case, it is possible to extract similar 
image data at higher speed by determining priorities 

25 for plural image data to be retrieved in advance and 
executing image retrieval in accordance with the 
priorities than executing image retrieval in an 
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arbitrary order. 

Thus, in this embodiment, priorities are 
determined for image data to be retrieved in advance 
and image retrieval is executed in accordance with the 
5 priorities. In other words, image data are sorted on 
the basis of incidental information such as 
photographing time, an amount of data, or photographing 
parameters of the image data to be retrieved and image 
retrieval is executed in accordance with an order of 

10 the sort. 

FIG. 25 is an exemplary flowchart of image 
retrieval processing according to this embodiment. The 
flowchart is equivalent to the subroutine of steps 
S2104 and S2309. 

15 In step S2501, the CPU 107 judges whether setting 

is made such that image retrieval is executed in 
accordance with priorities. When the image retrieval 
is executed in accordance with the priorities, the CPU 
107 proceeds to step S2502. Otherwise, the CPU 107 

20 proceeds to step S2504 and executes the image retrieval 
in an arbitrary order. 

In step S2502, the CPU 107 reads out a rule for 
determining priorities from the ROM 113 or the storage 
medium 114 . The rule is a rule for determining 

25 priorities for image data to be retrieved. Examples of 
the rule include a rule for performing image retrieval 
in order of lateness of photographing time, order of 
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earliness of photographing time, or order of closeness 
to photographing time of key image data, a rule for 
performing image retrieval in order of largeness of an 
amount of data, order of smallness of an amount of data, 
5 or order of closeness to an amount of data of key image 
data, a rule for performing image retrieval in order of 
largeness of photographing parameters, order of 
smallness of photographing parameters, or order of 
closeness to photographing parameters of key image data. 

10 In step S2503, the CPU 107 determines priorities 

of respective image data in accordance with the read- 
out rule. It is assumed that, for example, the CPU 107 
stores the determined priorities in the RAM 105 or the . 
storage medium 114 in a form of a table. 

15 In step S2504, the CPU 107 compares the image data 

with the key image data in order from the image data 
with a high priority and retrieves similar image data. 
As explained above, according to this embodiment, 
image retrieval is executed in accordance with 

20 priorities. Thus, efficiency of the image retrieval 

processing is improved compared with the case in which 
image retrieval is executed in an arbitrary order. 

In particular, in the case in which the CPU 107 
stops image retrieval when it is instructed from the 

25 operating unit 110 to stop the image retrieval during 
the image retrieval, an effect of image retrieval 
according to priorities is large. According to the 
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image retrieval according to priorities in this 
embodiment/ similar image data desired by a user is 
found in an initial stage of the image retrieval at a 
high probability. Therefore, there is little meaning 
5 in executing image retrieval on all the image data. 
Consequently, it is possible to end image retrieval 
earlier while finding desired image data if the image 
retrieval can be interrupted in the middle. This 
provides the user with a larger advantage. 

10 <Image display processing> 

FIG. 26 is an exemplary flowchart concerning 
display processing for a retrieval result according to 
this embodiment. The flowchart is equivalent to the 
subroutine in steps S2105 and S2310, 

15 In step S2601, the CPU 107 reads out data 

indicating the number of similar image data to be 
displayed at a time from the ROM 113 . Note that it is 
assumed that the number of similar image data to be 
displayed at a time is set through the operating unit 

20 110 in advance and stored in the ROM 113. For example, 
if the number of displayed images is reduced, since a 
display size of one similar image is increased, it is 
possible to perform detailed comparison. On the other 
hand, if the number of displayed images is increased, 

25 there is an effect that it is possible to evaluate a 

large number of similar images at a time while grasping 
a rough tendency of stored images. Note that. 
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alternatively, the number may always take a fixed value. 
In that case, step S2601 is unnecessary. 

In step S2602, the CPU 107 determines a display 
order of the retrieved image data. For example, the 
5 CPU 107 reads out a rule for determining a display 

order from the ROM 113 or the storage medium 114. The 
rule may be identical with the rule for determining 
priorities. Alternatively, the rule may be a rule that 
image data are displayed in descending order of 

10 similarity or ascending order of similarity. The CPU 

107 determines a display order of the retrieved similar 
image data in accordance with the read-out rule. The 
display order is also stored in the RAM 105 and the 
like in a foarm of a table. Note that this step is also 

15 optional. Usually, the image data may be displayed in 
an order of retrieval . 

In step S2603, the CPU 107 judges whether a 
display form of similar image data is set to icon 
display. It is assumed that information on whether the 

20 display form is set to icon display is inputted from 

the operating unit 110 in advance and stored in the ROM 
113 or the like. When icon display is performed, the 
CPU 107 proceeds to step S2608- Otherwise, the CPU 107 
proceeds to step S2604. Note that processing 

25 concerning the icon display is also optional. 

In step S2604, the CPU 107 judges whether a 
display form of similar image data and key image data 
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is set to emphatic display of a similar part. 
Information on whether a display form of similar image 
data and key image data is set to emphatic display is 
inputted from the operating unit 110 in advance and 
5 stored in the ROM 113 or the like. When emphatic 
display is performed, the CPU 107 proceeds to step 
S2606. otherwise, the CPU 107 proceeds to step S2605. 

In step S2605, the CPU 107 displays the key image 
data and the similar image data in parallel (e.g., FIG. 
10 22) , 

In the case of the emphatic display, in step S2606, 
the CPU 107 creates data for emphatic display. For 
example, for the similar image data and the key image 
data, the CPU 107 may create image data with a part of 

15 the respective images (e.g., similar parts) enlarged. 
The CPU 10 7 may create data for displaying a part of 
the images (e.g., similar parts) in blinking. The CPU 
107 may create data for displaying the images with a 
part of the images (e.g., similar parts) encircled with 

20 a line. Alternatively, the CPU 107 may create data for 
changing a color of a part of the images (e.g., similar 
parts) . Note that, when the imaging apparatus 100 has 
plural emphatic display methods in advance, it is 
possible to perform emphatic display according to a 

25 preference of a user by allowing the user to select any 
one of the emphatic display methods from the operating 
unit 110 or the like. If the CPU 107 emphatically 
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displays a part of the images that allows the user to 
judge hand shake, eye close, white void, and the like, 
this is useful in deleting an unnecessary image. 

In step S2607, the CPU 107 causes the display unit 
5 109 to execute emphatic display using the key image 

data, the similar image data, and the data for emphatic 
display. 

FIG. 27 is a diagram showing an example of display 
of a retrieval result according to this embodiment. In 

10 this example, parts 2700 indicating a similar 
characteristic are encircled by lines to be 
emphatically displayed. Usually, since the display 
unit 109 provided in a digital camera is a liquid 
crystal display of several inches, it is difficult to 

15 evaluate plural images simultaneously on the display 
unit 109. Therefore, the evaluation is made easier by 
emphasizing a part of the images. 

It is also possible that presence of a similar 
image data can be notified to a user by displaying an 

20 icon instead of directly displaying the similar image 
data . 

In step S2608, the CPU 107 reads out icon data 
corresponding to the similar image data from the ROM 
113. As the icon, there are an icon indicting 
25 similarity and an icon indicating a similar 

characteristic. The similar image retrieving unit 112 
or the CPU 10 7 determines the similarity and the 
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similar characteristic on the basis of the key image 
data and the similar image data. The CPU 107 reads out 
icon data corresponding to the determined similarity 
and characteristic. The CPU 107 may create icon data 
5 every time instead of reading out icon data. 

Concerning the similarity, for example, icons 
formed by digitizing similarity in several stages (e.g., 
five stages) or icons imitating alphsGDets representing 
similarity may be formed. Concerning the similar 

10 characteristic, icons imitating human faces may be 
formed when people on images are similar, icons 
imitating a mountain, a river, and the like may be 
formed when sceneries are similar, or icons indicating 
colors such as red and blue may be formed when color 

15 distributions are similar. In addition, concerning 

icons indicating the similar characteristic, character 
icons such as "person in Chinese character", "scenery 
in Chinese character", "color in Chinese character", 
and the like may be formed using one Chinese character 

20 corresponding to the similar characteristic. 

In step S2609, the CPU 107 displays an icon 
corresponding to the similar image data together with 
the key image data . 

FIG. 2 8 is a diagram showing an example of display 

25 of icons according to this embodiment. In this example, 
a retrieval result of similar image data is displayed 
with character icons 2800 on key image data 2200. Even 
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if the display unit 109 is small, it is possible to 
cause a user to recognize presence of a similar image 
without preventing display of a preview image or a 
photographed image by displaying presence of similar 
5 image data and similar points thereof using icons. 

In step S2610, the CPU 107 selects image data to 
be deleted according to a selection instruction from 
the operating unit 110. 

In step S2611, the CPU 107 deletes the selected 
10 image data from the storage medium 114 or the RAM 105. 
Unnecessary images can be deleted easily by making it 
possible to delete any one of image data when the 
retrieval result is displayed on the display unit 109 
in this way. 

15 In an imaging apparatus such as a digital camera, 

since it is possible to perform photographing easily 
compared with a film camera, unnecessary images tend to 
be accumulated. For example, since a user can inspect 
an image immediately after photographing the image, if 

20 the user is dissatisfied with a photographing result, 
the user perfoarms photographing again under a similar 
situation immediately. This also causes the 
accumulation of unnecessary images. In addition, if 
the user performs photographing when the user fails to 

25 remember that the user photographed a similar image 

before, similar image data are accumulated. If similar 
image data can be deleted appropriately in such a case. 



I 



P205-0195WO 

- 84 - 

the user can sort out image data that the user owns . 

As explained above, according to this embodiment, 
even in an imaging apparatus with an insufficient size 
of a display unit, it becomes easy to cause a user to 
5 recognize presence of similar images by displaying 
icons . 

In addition, since the emphatic display is used, 
it becomes easy to evaluate similar image data even in 
a digital camera with an insufficient size of a display 
10 unit. 

Moreover, since photographed image data and 
similar image data thereof are displayed in parallel, a 
user can easily recognize and delete unnecessary data. 
< Image print process ing> 

15 In this embodiment, retrieval processing for 

similar image data and display of a retrieval result 
are performed in the case of so-called direct print in 
which printing is performed by directly connecting an 
imaging apparatus and an image forming apparatus (e.g., 

20 a printer, a copying machine, a multifunctional product, 
etc.) without the intervention of a PC. It is needless 
to mention that the retrieval result may be printed by 
the direct print . 

In printing image data, paper dedicated for 

25 photographs, which is different from print paper used 
for usual printing, is often used. The paper dedicated 
for photographs is more expensive than the usual print 
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paper. Therefore, it is not only useless but also 
wasteful to print similar image data on many sheets. 
It is conceivable to reduce such a waste by executing 
retrieval processing for similar image data before 
5 printing. In other words, it is possible to control 
the uselessness of executing print processing on for 
similar image data on many sheets. 

FIG. 29 is a diagram showing an example of a 
direct print system according to this embodiment . In 

10 this example, the communication unit 106 of the imaging 
apparatus 100 and a printer 2900 are connected by a 
connection cable 2901. As a connection system, for 
example, not only a wire system such as USB or IEEE13 94 
but also a wireless system such as Bluetooth, wireless 

15 LAN, or IrDA may be adopted. The communication unit 

106 includes a communication circuit, a connector, and 
the like conforming to at least one of these systems. 

FIG. 3 0 is an exemplary flowchart of direct print 
processing according to this embodiment. In this 

20 example, the imaging apparatus 100 and the printer 2900 
are connected directly without the intervention of a PC, 
It is assumed that print setting necessary for the 
direct print has already been executed. 

In step S3001, the CPU 107 selects image data, 

25 which a user desires to print, according to a selection 
instruction from the operating unit 110. 

In step S3002, the CPU 107 judges whether setting 
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is made such that retrieval for a similar image is 
executed at the time of the direct print. Note that it 
is assumed that the CPU 107 stores setting information 
of the setting in the ROM 113 according to a setting 
5 instruction from the operating unit 110 in advance. 

For example, it is assumed that a user sets the setting 
information using a menu displayed on the display unit 
109 and the operating unit 110. If the setting 
information is not set such that retrieval for similar 

10 image data is executed, the CPU 107 proceeds to step 
S3 009 and transmits an instruction for printing 
selected image data (print job data) to the printer 
2900 through the communication unit 106. On the other 
hand, if the setting information is set such that 

15 retrieval for similar image data is executed, the CPU 
107 proceeds to step S3003. 

In step S3003, the CPU 107 retrieves similar image 
data. A specific retrieval method is as explained in 
steps S2104 and S2309. 

20 In step S3004, the CPU 107 judges whether the 

retrieved similar image data should be displayed on the 
display unit 109 or printed immediately without being 
displayed. Note that it is assumed that, through the 
operating unit 110, it is set in advance which 

25 processing should be selected in the same manner as the 
setting for the setting information explained in step 
S3002. Alternatively, the CPU 107 may display an 
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inquiry message on the display xinit 109 and judge 
whether the similar image data should be displayed or 
printed according to an operation instruction from the 
operating unit 110. When it is judged that the similar 
5 image data- should be printed, the CPU 107 proceeds to 
step S3008 and adds the retrieved similar image data as 
a print object. On the other hand, when it is judged 
that the similar image data should be displayed, the 
CPU 107 proceeds to step S3005. 
10 In step S3005, the CPU 107 displays the retrieved 

similar image data on the display unit 109. Specific 
display processing is as explained in steps S2105 and 
S2310 . 

In step S3006, the CPU 107 judges whether setting 
15 is made such that image data, which is finally printed, 
is selected out of the key image data (in this example, 
the image data selected in step S3001) and the 
retrieved similar image data. It is assumed that 
contents of the setting are also set in advance as 
20 described above and the CPU 107 stores the contents in 
the ROM 113 as setting information. When setting is 
made such that a final print object is selected, the 
CPU 107 proceeds to step S3007. Otherwise, the CPU 107 
proceeds to step S3009. Note that the CPU 107 may 
25 display an inquiry message on the display unit 109 and 
execute the judgment according to selection information 
from the operating unit 110. 
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In step S3007, the CPU 107 selects image data to 
be printed last out of the key image data (in this 
example, the image data selected in S3 001) and the 
retrieved similar image data according to the selection 
5 instruction from the operating unit 110. 

In step S3009, the CPU 107 executes print 
processing according to the direct print on the 
selected image data. 

In an example of printing of a retrieval result 

10 shown in FIG. 29, print object image data 2 910 and 
similar image data 2930 similar to the print object 
image data 2910 are printed on a print medium 2920. 

According to this embodiment, in the direct print, 
a user can search for image data similar to image data 

15 to be printed and check the image data. Therefore, for 
example, when similar image data has already been 
printed, the user can stop printing according to the 
user's own judgment. Thus, it is possible to control 
useless printing. 

2 0 In general, a print medium is larger than a size 

of the display unit 109. Thus, if all of retrieved 
similar image data are printed or a part of the similar 
image data are selected and printed, since visibility 
is improved, the user can easily compare images. 

25 In image retrieval, if the image retrieval 

according to priorities is used and a retrieval result 
thereof is printed by the direct print, it is possible 
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to perform printing according to the user's desire for 
inspection. 

The number of images to be printed on one storage 
medium may be instructed from the operating \init 110. 
5 For example, if the number of images to be printed is 
reduced, it is possible to compare similar images in 
detail. On the other hand, if the number of images is 
increased, it is possible not only to learn a rough 
tendency of similar images but also to reduce the 
10 number of storage medium required for printing. In 

particular, if images are printed thumbnail images, it 
is possible to judge an outline of similar images and a 
tendency of similarity at a time. 

Steps S3002, S3004, and S3006 described above are 
15 optional. None of the steps may be adopted or one or 
more of the steps may be adopted. 
[Other embodiment s ] 

The various embodiments have been described in 
detail. However, the invention may be applied to a 
20 system including plural apparatuses or may be applied 
to an apparatus consisting of one device. 

Plural image retrieval processings and display 
processing and print processing for a retrieval result 
may be combined in any way. In other words, it is 
25 needless to mention that inventions corresponding to 
all combinations are disclosed herein. 

In executing retrieval for a similar image, it is 
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also possible that information on whether the retrieval 
is executed is stored in the ROM 113 as setting 
information in advance and the CPU 107 judges whether 
the retrieval should be executed on the basis of the 
5 setting information every time the retrieval is 
executed. It is possible to set the setting 
information from a menu displayed on the display* unit 
109 by operating the operating unit 110. Setting work 
is easier if the operating unit 110 includes a cross 

10 key or buttons . 

The CPU 107 can stop retrieval processing for a 
similar image in the middle of retrieval according to 
an instruction from the operating unit 110. When the 
retrieval processing is stopped in the middle, the CPU 

15 107 displays a retrieval result at that stage on the 
display unit 109. Note that retrieval at the time of 
direct print may also be stopped in the middle. 

If the retrieval processing for similar image data 
is executed when a panoramic image is created, it is 

20 possible to retrieve similar image data, from which 
panoramic image data can be formed, by connecting 
similar image data to photographed image data (key 
image data) . In other words, the CPU 107 creates 
panoramic image data by connecting the photographed 

25 image data and the similar image data side by side or 
superimposing a part of the photographed image data on 
a part of the similar image data. Thus, a user can 
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execute panoramic photographing without being conscious 
that the panoramic photographing is performed. 

Instead of the method of displaying a retrieval 
result, the CPU 107 may create thumbnail image data of 
5 similar image data and display the thumbnail image data 
on the display unit 109. Since detailed information is 
not included in a thumbnail image, the thumbnail image 
is not preferable in terms of evaluation of a similar 
image but has an advantage that a retrieval result can 

10 be displayed at high speed. Note that in the case of 
the direct print, the CPU 107 may subject thumbnail 
image data to print processing. 

In addition, a software program for realizing the 
respective functions of the embodiments (in this 

15 embodiment, programs corresponding to the flowcharts 
shown in FIGS. 4 to 9, FIGS. 13 to 17, FIG. 21, FIGS. 
23 to 26, or FIG. 30) is supplied to a system or an 
apparatus directly or from a remote location. A 
computer included in the system or the apparatus reads 

20 out and executes a supplied program code. In this way, 
the invention is also attained. 

Therefore, the program code itself installed in 
the computer in order to realize the functions and the 
processing of the invention also realizes the invention. 

25 In other words, the computer program itself for 

realizing the functions and processing is a part of the 
invention. 
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In that case, the program may take any form such 
as an object code, a program executed by an interpreter, 
and script data supplied to an OS as long as the 
program has a function as a program. 
5 As storage medium for supplying the program, for 

example, there are a flexible disk, a hard disk, an 
optical disk, a magneto-optical disk, a CD-ROM, a CD-R, 
a CD-RW, a magnetic tape, a nonvolatile memory card, a 
ROM, and a DVD (a DVD-ROM, a DVD-R) . 

10 As a method of supplying the program, it is 

possible to supply the program by making connection to 
a web page on the Internet using a browser of a client 
computer and downloading a computer program itself of 
the invention or a file, which is compressed and 

15 includes an automatic install function, to a storage 
medium such as a hard disk from the web page. In 
addition, it is also possible to realize the invention 
by dividing a program code forming the program of the 
invention into plural files and downloading the 

20 respective files from different web pages. In other 
words, a WWW server, which causes plural users to 
download a program file for realizing the functional 
processing of the invention with a computer, may be an 
element of the invention. 

25 The program of the invention is encrypted and 

stored in a storage medium such as a CD-ROM and 
distributed to users. A user meeting predetermined 
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conditions is allowed to download key information for 
decryption from a web page through the Internet. The 
user executes the encrypted program using the key 
information and installs the program in a computer. It 
5 is also possible to realize the invention in this way. 

The computer executes a read- out program, whereby 
the functions of the embodiments are realized. In 
addition, an OS or the like running on the computer 
performs a part or all of actual processings. The 

10 functions of the embodiments can be realized by the 
processings . 

A program read out from the storage medium is 
written in memory provided in a function extended board, 
inserted in the computer or a function extended unit 

15 connected to the computer. Then, a CPU or the like 

provided in the function extended board or the function 
extended unit performs a part or all of actual 
processings on the basis of an instruction of the 
program. The functions of the embodiments are also 

20 realized by the processings. 

As many apparently widely different embodiments of 
the present invention can be made without departing 
from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the 

25 specific embodiments thereof except as defined in the 
appended claims. 



